Learning real-world stimuli in a neural network with spike-driven synaptic dynamics

Neural Computation

Volumn 19, Issue 11, 2007, Pages 2881-2912

  Brader, Joseph M ^a   Senn, Walter ^a   Fusi, Stefano ^a,b  

^a UNIVERSITY OF BERN   (Switzerland)

^b ETH ZURICH   (Switzerland)

Author keywords

[No Author keywords available]

Indexed keywords

CALCIUM;

ACTION POTENTIAL; ANIMAL; ARTICLE; ARTIFICIAL NEURAL NETWORK; BIOLOGICAL MODEL; COMPUTER SIMULATION; LEARNING; METABOLISM; NERVE CELL; NERVE CELL PLASTICITY; NONLINEAR SYSTEM; PHYSIOLOGY; PROBABILITY; SYNAPSE; SYNAPTIC TRANSMISSION; TIME;

ACTION POTENTIALS; ANIMALS; CALCIUM; COMPUTER SIMULATION; LEARNING; MODELS, NEUROLOGICAL; NEURAL NETWORKS (COMPUTER); NEURONAL PLASTICITY; NEURONS; NONLINEAR DYNAMICS; PROBABILITY; SYNAPSES; SYNAPTIC TRANSMISSION; TIME FACTORS;

EID: 36248934673     PISSN: 08997667     EISSN: 1530888X     Source Type: Journal    
DOI: 10.1162/neco.2007.19.11.2881     Document Type: Article

References (78)

1. Abarbanel, H. D. I., Huerta, R., & Rabinovich, M. I. (2002). Dynamical model of long-term synaptic plasticity. PNAS, 99, 10132-10137.
2. Abbott, L. F., & Nelson, S. B. (2000). Synaptic plasticity: Taming the beast. Nature Neuroscience, 3, 1178-1183.
3. Adini, Y., Sagi, D., & Tsodyks, M. (2002). Context enabled learning in human visual system. Nature, 415, 790-794.
4. Albus, J. (1971). A theory of cerebellar function. Math. Biosci., 10, 26-51.
5. Amit, D. (1989). Modeling brain function. Cambridge: Cambridge University Press.
6. Amit, D. J., & Fusi, S. (1992). Constraints on learning in dynamic synapses. Network, 3, 443.
7. Amit, D., & Fusi, S. (1994). Learning in neural networks with material synapses. Neural Comput., 6(5), 957-982.
8. Amit, Y., & Geman, D. (1997). Shape quantization and recognition with randomized trees. Neural Computation, 9, 1545-1588.
9. Amit, Y., & Mascaro, M. (2001). Attractor networks for shape recognition. Neural Computation, 13, 1415-1442.
10. Amit, D., & Mongillo, G. (2003). Spike-driven synaptic dynamics generating working memory states. Neural Computation, 15, 565-596.
11. Badoni, D., Giulioni, Dante, V., & Del Giudice, P. (2006). A VLSI recurrent network of spiking neurons with reconfigurable and plastic synapses. IEEE International Symposium on Circuits and Systems ISCAS06 (pp. 1227-1230). Piscataway, NJ: IEEE.
12. Belongie, S., Malik, J., & Puzicha, J. (2002). Shape matching and object recognition using shape context. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(24), 509.
13. Block, H. (1962). The perceptron: A model for brain functioning. I. Reviews of Modern Physics, 34, 123-135.
14. Boahen, K. (1998). Communicating neuronal ensembles between neuromorphic chips. In S. Smith & A. Hamilton (Eds.), Neuromorphic systems engineering. Norwell, MA: Kluwer.
15. Brunel, N., Carusi, F., & Fusi, S. (1998). Slow stochastic Hebbian learning of classes of stimuli in a recurrent neural network. Network, 9(1), 123-152.
16. Brunel, N., Hakim, V., Isope, P., Nadal, J.-P., & Barbour, B. (2004). Optimal information storage and the distribution of synaptic weights: Perceptron versus Purkinje cells. Neuron, 43, 745-757.
17. Buchs, N., & Senn, W. (2002). Spike-based synaptic plasticity and the emergence of direction selective simple cells: Simulation results. Journal of Computational Neuroscience, 13, 167-186.
18. Burkitt, A. N., Meffin, H., & Grayden, D. B. (2004). Spike timing-dependent plasticity: The relationship to rate-based learning for models with weight dynamics determined by a stable fixed-point. Neural Computation, 16, 885-940.
19. Chicca, E., & Fusi, S. (2001). Stochastic synaptic plasticity in deterministic aVLSI networks of spiking neurons. In F. Rattay (Ed.), Proceedings of the World Congress on Neuroinformatics (pp. 468-477). Verlag, Vienna: ARGESIM/ASIM.
20. Chicca, E., Indiveri, G., & Douglas, R. (2003). An adaptive silicon synapse. IEEE International Symposium on Circuits and Systems. Piscataway, NJ: IEEE Press.
21. Coath, M., Brader, J., Fusi, S., & Denham, S. (2005). Multiple views of the response of an ensemble of spectro-temporal features support concurrent classification of utterance, prosody, sex and speaker identity. NETWORK Computation in Neural Systems, 16, 285-300.
22. Diorio, C., Hasler, P., Minch, B., & Mead, C. (1996). A single-transistor silicon synapse. IEEE Transactions on Electron Devices, 43, 1972-1980.
23. Douglas, R., Deiss, S., & Whatley, A. (1998). A pulse-coded communications infrastructure for neuromorphic systems. In W. Maass & C. Bishop (Eds.), Pulsed neural networks (pp. 157-178). Cambridge, MA: MIT Press.
24. Feldman, D. (2000). Abstract timing-based LTP and LTD at vertical inputs to layer II/III pyramidal cells in rat barrel cortex. Neuron, 27, 45-56.
25. Freund, Y., & Schapire, R. (1999). A short introduction to boosting. Journal of the Japanese Society for Artificial intelligence, 14, 771-780.
26. Froemke, R., & Dan, Y. (2002). Spike-timing-dependent synaptic modification induced by natural spike trains. Nature, 416, 433-438.
27. Fusi, S. (2001). long-term memory: Encoding and storing strategies of the brain. In J. M. Bower (Ed.), Neurocomputing: Computational neuroscience: Trends in research 2001 (Vol. 38-40, pp. 1223-1228). Amsterdam: Elsevier Science.
28. Fusi, S. (2002). Hebbian spike-driven synaptic plasticity for learning patterns of mean firing rates. Biol. Cybern., 87, 459.
29. Fusi, S. (2003). Spike-driven synaptic plasticity for learning correlated patterns of mean firing rates. Rev. Neurosci., 14, 73-84.
30. Fusi, S., & Abbott, L. (2007). Limits on the memory-storage capacity of bounded synapses. Nature Neuroscience, 10, 485-493.
31. Fusi, S., Annunziato, M., Badoni, D., Salamon, A., & Amit, D. (2000). Spike-driven synaptic plasticity: Theory, simulation, VLSI implementation. Neural Computation, 12, 2227-2258.
32. Fusi, S., Drew, P., & Abbott, L. (2005). Cascade models of synaptically stored memories. Neuron, 45, 599-611.
33. Fusi, S., & Mattia, M. (1999). Collective behavior of networks with linear (VLSI) integrate-and-fire neurons. Neural Comput., 11(3), 633-653.
34. Fusi, S., & Senn, W. (2006). Eluding oblivion with smart stochastic selection of synaptic updates. Chaos, 16, 026112.
35. Gerstner, W., Kempter, R., van Hemmen, J., & Wagner, H. (1996). A neuronal learning rule for sub-millisecond temporal coding. Nature, 383, 76-78.
36. Giudice, P. D., Fusi, S., & Mattia, M. (2003). Modeling the formation of working memory with networks of integrate-and-fire neurons connected by plastic synapses. Journal of Physiology, Paris, 97, 659-681.
37. Giudice, P. D., & Mattia, M. (2001). Long and short-term synaptic plasticity and the formation of working memory: A case study. Neurocomputing, 38-40, 1175-1180.
38. Gutig, R., & Sompolinsky, H. (2006). The tempotron: A neuron that learns spike timing-based decisions. Nat. Neurosci., 9(3), 420-428.
39. Hertz, J., Krogh, A., & Palmer, R. (1991). Introduction to the theory of neural computation. Reading, MA: Addison-Wesley.
40. Hopfield, J. J. (1982). Neural networks and physical systems with emergent selective computational abilities. Proc. Natl. Acad. Sci. USA, 79, 2554.
41. Hopfield, J., & Brody, C. D. (2004). Learning rules and network repair in spike-timing-based computation networks. PNAS, 101, 337-342.
42. Indiveri, G. (2000). Modeling selective attention using a neuromorphic AVLSI device. Neural Comput., 12, 2857-2880.
43. Indiveri, G. (2001). A neuromorphic VLSI device for implementing 2D selective attention systems. IEEE Transactions on Neural Networks, 12, 1455-1463.
44. Indiveri, G. (2002). Neuromorphic bistable VLSI synapses with spike-timing-dependent plasticity. In S. Becker, S. Thrün, & K. Obermayer (Eds.), Advances in neural information processing systems, 15. Cambridge, MA: MIT Press.
45. Indiveri, G., & Fusi, S. (2007). Spike-based learning in VLSI networks of spiking neurons. In Proceedings of the IEEE International Symposium on Circuits and Systems (pp. 3371-3374). Piscataway, NJ: IEEE Press.
46. Karmarkar, U., & Buonomano, D. (2002). A model of spike-timing dependent plasticity: one or two coincidence detectors? J. Neurophysiology, 88, 507-513.
47. Kempter, R., Gerstner, W., & van Hemmen, J. (2001). Intrinsic stabilization of output firing rates by spike-based Hebbian learning. Neural Computation, 13, 2709-2741.
48. La Camera, G., Rauch, A., Lüscher, H.-R., Senn, W., & Fusi, S. (2004). Minimal models of adapted neuronal response to in vivo-like input currents. Neural Comput., 16, 2101-2124.
49. Legenstein, R., Naeger, C., & Maass, W. (2005). What can a neuron learn with spike-timing-dependent plasticity? Neural Comput, 17(11), 2337-2382.
50. Markram, H., Lübke, J., Frotscher, M., & Sakmann, B. (1997). Regulation of synaptic efficacy by coindence of postsynaptic APs and EPSPs. Science, 275, 213-215.
51. Marr, D. (1969). A theory of cerebellar cortex. J. Physiol., 202, 435-170.
52. Minsky, M. L., & Papert, S. A. (1969). Perceptrons. Cambridge, MA: MIT Press.
53. Mitra, S., Fusi, S., & Indiveri, G. (2006). A VLSI spike-driven dynamic synapse which learns only when necessary. IEEE International Symposium on Circuits and Systems ISCAS06 (pp. 2777-2780). Piscataway, NJ: IEEE Press.
54. Mongillo, G., Curti, E., Romani, S., & Amit, D. (2005). Learning in realistic networks of spiking neurons and spike-driven plastic synapses. European Journal of Neuroscience, 21, 3143-3160.
55. Nadal, J. P., Toulouse, G., Changeux, J. P., & Dehaene, S. (1986). Networks of formal neurons and memory palimpsests. Europhys. Lett., 1, 535.
56. Parisi, G. (1986). A memory which forgets. J. Phys. A: Math. Gen., 19, L617.
57. Petersen, C. C. H., Malenka, R. C., Nicoll, R. A., & Hopfield, J. J. (1998). All-or-none potentiation at CA3-CA1 synapses. Proc. Natl. Acad. Sci. USA, 95, 4732-4737.
58. Rao, R. P. N., & Sejnowski, T. J. (2001). Spike-timing-dependent Hebbian plasticity as temporal difference learning. Neural Computation, 13, 2221-2237.
59. Rauch, A., La Camera, G., Lüscher, H.-R., Senn, W., & Fusi, S. (2003). Neocortical pyramidal cells respond as integrate-and-fire neurons to in vivo-like input currents. J. Neurophysiology, 90, 1598-1612.
60. Rosenblatt, F. (1958). The perceptron: A probabilistic model for information storage and organization in the brain. Psychological Review, 65, 386-408.
61. Rubin, J., Lee, D., & Sompolinsky, H. (2001). The equilibrium properties of temporally asymmetric Hebbian plasticity. Physical Review Letters, 86, 364-367.
62. Saudargiene, A., Porr, B., & Wörgötter, F. (2004). How the shape of pre- and postsynaptic signals can influence STDP: A biophysical model. Neural Computation, 16, 595-625.
63. Senn, W., & Buchs, N. (2003). Spike-based synaptic plasticity and the emergence of direction selective simple cells: Mathematical analysis. Journal of Computational Neuroscience, 14, 119-138.
64. Senn, W., & Fusi, S. (2005a). Convergence of stochastic learning in perceptrons with binary synapses. Phys. Rev. E., 71, 061907.
65. Senn, W., & Fusi, S. (2005b). Learning only when necessary: Better memories of correlated patterns in networks with bounded synapses. Neural Computation, 17, 2106-2138.
66. Senn, W., Markram, H., & Tsodyks, M. (2001). An algorithm for modifying neurotransmitter release probability based on pre- and postsynaptic spike timing. Neural Comput., 13(1), 35-67.
67. Shouval, H., & Kalantzis, G. (2005). Stochastic properties of synaptic transmission affect the shape of spike time dependent plasticity curves. J. Neurophysiology, 93, 643-655.
68. Shouval, H. Z., Bear, M. F., & Cooper, L. N. (2002). A unified model of NMDA receptor-dependent bidirectional synaptic plasticity. PNAS, 99, 10831-10836.
69. Simard, P., Steinkraus, D., & Platt, J. (2003). Best practice for convolutional neural networks applied to visual document analysis. In Proceedings of the Seventh International Conference on Document Analysis and Recogntion (p. 958). Washington, DC: IEEE Computer Society.
70. Sjöström, P. J., Turrigiano, G. G., & Nelson, S. B. (2001). Rate, timing and co-operativity jointly determine cortical synaptic plasticity. Neuron, 32, 1149-1164.
71. Tsodyks, M. (1990). Associative memory in neural networks with binary synapses. Mod. Phys. Lett., B4, 713-716.
72. Turrigiano, G., & Nelson, S. (2000). Hebb and homeostasis in neuronal plasticity. Curr. Opin. Neurobiol., 10, 358-364.
73. Wang, H., & Wagner, J. (1999). Priming-induced shift in synaptic plasticity in the rat hippocampus. J Neurophysiol., 82, 2024-2028.
74. Wang, S., O'Connor, D., & Wittenberg, G. (2004). Steplike unitary events underlying bidirectional hippocampal synaptic plasticity. Society for Neuroscience, p. 57.6.
75. Wang, X.-J. (2002). Probabilistic decision making by slow reverbaration in cortical circuits. Neuron, 36, 955-968.
76. Yao, H., Shen, Y., & Dan, Y. (2004). Intracortical mechanism of stimulus-timing-dependent plasticity in visual cortical orientation tuning. PNAS, 101, 5081-5086.
77. Zhang, L. I., Tao, H. W., Holt, C. E., Harris, W. A., & Poo, M. (1998). A critical window for cooperation and competition among developing retinotectal synapses. Nature, 395, 37-44.
78. Zhou, Q., Tao, H., & Poo, M. (2003). Reversal and stabilization of synaptic modifications in a developing visual system. Science, 300, 1953.