Optimality model of unsupervised spike-timing-dependent plasticity: Synaptic memory and weight distribution

Neural Computation

Volumn 19, Issue 3, 2007, Pages 639-671

  Toyoizumi, Taro ^a,b   Pfister, Jean Pascal ^a   Aihara, Kazuyuki ^c,d   Gerstner, Wulfram ^a  

^a EPFL   (Switzerland)

^b UNIVERSITY OF TOKYO   (Japan)

^c INSTITUTE OF INDUSTRIAL SCIENCE   (Japan)

^d JAPAN SCIENCE AND TECHNOLOGY AGENCY   (Japan)

Author keywords

[No Author keywords available]

Indexed keywords

ACTION POTENTIAL; ANIMAL; ARTICLE; BIOLOGICAL MODEL; CYTOLOGY; NERVE CELL; NERVE CELL PLASTICITY; NONLINEAR SYSTEM; PHYSIOLOGY; REACTION TIME; STANDARD; STATISTICS; SYNAPSE; SYNAPTIC TRANSMISSION; TIME;

ACTION POTENTIALS; ANIMALS; MODELS, NEUROLOGICAL; NEURONAL PLASTICITY; NEURONS; NONLINEAR DYNAMICS; REACTION TIME; STOCHASTIC PROCESSES; SYNAPSES; SYNAPTIC TRANSMISSION; TIME FACTORS; WEIGHTS AND MEASURES;

EID: 33847632899     PISSN: 08997667     EISSN: 1530888X     Source Type: Journal    
DOI: 10.1162/neco.2007.19.3.639     Document Type: Article

References (75)

1. Abarbanel, H., Talathi, S., Gibb, L., & Rabinovich, M. (2005). Synaptic plasticity with discrete state synapses. Physical Review E, 72, art. no. 031917 Part 1.
2. Abraham, W., Logan, B., Greenwood, J., & Dragunow, M. (2002). Induction and experience-dependent consolidation of stable long-term potentiation lasting months in the hippocampus. J. Neuroscience, 22, 9626-9634.
3. Amit, D., & Fusi, S. (1994). Learning in neural networks with material synapses. Neural Computation, 6, 957-982.
4. Atick, J., & Redlich, A. (1990). Towards a theory of early visual processing. Neural Computation, 4, 559-572.
5. Barlow, H. (1956). Retinal noise and absolute threshold. J. Opt. Soc. Am., 46, 634-639.
6. Barlow, H. B. (1961). Possible principles underlying the transformation of sensory messages. In W. A. Rosenbluth (Ed.), Sensory communication (pp. 217-234). Cambridge, MA: MIT Press.
7. Bell, A. J., & Parra, L. C. (2005). Maximising sensitivity in a spiking network. In L. K. Saul, Y. Weiss, & L. Bottou (Eds.), Advances in neural information processing systems, 17 (pp. 121-128). Cambridge, MA: MIT Press.
8. Bell, A., & Sejnowski, T. (1995). An information maximization approach to blind separation and blind deconvolution. Neural Computation, 7, 1129-1159.
9. Bi, G., & Poo, M. (1998). Synaptic modifications in cultured hippocampal neurons: Dependence on spike timing, synaptic strength, and postsynaptic cell type. J. Neurosci., 18, 10464-10472.
10. Bi, G., & Poo, M. (2001). Synaptic modification of correlated activity: Hebb's postulate revisited. Ann. Rev. Neurosci., 24, 139-166.
11. Bienenstock, E., Cooper, L., & Munroe, P. (1982). Theory of the development of neuron selectivity: Orientation specificity and binocular interaction in visual cortex. Journal of Neuroscience, 2, 32-48.
12. Bliss, T. V. P., & Collingridge, G. L. (1993). A synaptic model of memory: Long-term potentiation in the hippocampus. Nature, 361, 31-39.
13. Bliss, T., & Lomo, T. (1973). Long-lasting potentation of synaptic transmission in the dendate area of anaesthetized rabbit following stimulation of the perforant path. J. Physiol., 232, 351-356.
14. Bohte, S. M., & Mozer, M. C. (2005). Reducing spike train variability: A computational theory of spike-timing dependent plasticity. In L. K. Saul, Y. Weiss, & L. Bottou (Eds.), Advances in neural information processing systems, 17 (pp. 201-208). Cambridge, MA: MIT Press.
15. Bortolotto, Z. A., Lauri, S., Isaac, J. T. R., & Collingridge, G. L. (2003). Kainate receptors and the induction of mossy fibre long-term potentiation. Phil Trans. R. Soc. Lond B: Biological Sciences, 358, 657-666.
16. Britten, K., Shadlen, M., Newsome, W., & Movshon, J. (1992). The analysis of visual motion: A comparison of neuronal and psychophysical performance. J. Neuroscience, 12, 4745-4765.
17. Brunel, N., Hakim, V., Isope, P., Nadal, J.-P., & Barbour, B. (2004). Optimal information storage and the distribution of synaptic weights: Perceptron versus Purkinje cell. Neuron, 43, 745-757.
18. Buonomano, D., & Merzenich, M. (1998). Cortical plasticity: From synapses to maps. Annual Review of Neuroscience, 21, 149-186.
19. Chechik, G. (2003). Spike-timing-dependent plasticity and relevant mututal information maximization. Neural computation, 15, 1481-1510.
20. Cooper, L., Intrator, N., Blais, B., & Shouval, H. Z. (2004). Theory of cortical plasticity. Singapore: World Scientific.
21. Cover, T., & Thomas, J. (1991). Elements of information theory. New York: Wiley.
22. Dayan, P., & Abbott, L. F. (2001). Theoretical nuroscience. Cambridge, MA: MIT Press.
23. de Ruyter van Steveninck, R. R., & Bialek, W. (1995). Reliability and statistical efficiency of a blowfly movement-sensitive neuron. Phil. Trans. R. Soc. Lond. Ser. B, 345, 321-340.
24. Dudek, S. M., & Bear, M. F. (1992). Homosynaptic long-term depression in area CA1 of hippocampus and effects of N-methyl-D-aspartate receptor blockade. Proc. Natl. Acad. Sci. USA, 89, 4363-4367.
25. Erwin, E., Obermayer, K., & Schulten, K. (1995). Models of orientation and ocular dominance columns in the visual cortex: A critical comparison. Neural Comput., 7, 425-468.
26. Fonseca, R., Nägerl, U., Morris, R., & Bonhoeffer, T. (2004). Competition for memory: Hippocampal LTP under the regimes of reduced protein synthesis. Neuron, 44, 1011-1020.
27. Frey, U., & Morris, R. (1997). Synaptic tagging and long-term potentiation. Nature, 355, 533-536.
28. Froemke, R. C., Poo, M.-M., & Dan, Y. (2005). Spike-timing-dependent synaptic plasticity depends on dendritic location. Nature, 434, 221-225.
29. Fusi, S. (2002). Hebbian spike-driven synaptic plasticity for learning patterns of mean firing rates. Biol. Cybem., 87, 459-470.
30. Fusi, S., Annunziato, M., Badoni, D., Salamon, A., & Amit, D. J. (2000). Spike-driven synaptic plasticity: Theory, simulation, VLSI implementation. Neural Computation, 12, 2227-2258.
31. Fusi, S., Drew, P., & Abbott, L. (2005). Cascade models of synaptically stored memories. Neuron, 45, 599-611.
32. Gerstner, W., Kempter, R., van Hemmen, J. L., & Wagner, H. (1996). A neuronal learning rule for sub-millisecond temporal coding. Nature, 383, 76-78.
33. Gerstner, W., & Kistler, W. K. (2002). Spiking neuron models. Cambridge: Cambridge University Press.
34. Grossberg, S. (1987). The adaptive brain I. Dordrecht: Elsevier.
35. Gutig, R., Aharonov, R., Rotter, S., & Sompolinsky, H. (2003). Learning input correlations through non-linear temporally asymmetry Hebbian plasticity. J. Neuroscience, 23, 3697-3714.
36. Hebb, D. O. (1949). The organization of behavior. New York: Wiley.
37. Hertz, J., Krogh, A., & Palmer, R. G. (1991). Introduction to the theory of neural computation. Redwood City, CA: Addison-Wesley.
38. Hubel, D. H., & Wiesel, T. N. (1962). Receptive fields, binocular interaction and functional architecture in the cat's visual cortex. J. Physiol. (London), 160, 106-154.
39. Katz, L. C., & Shatz, C. J. (1996). Synaptic activity and the construction of cortical circuits. Science, 274, 1133-1138.
40. Kelso, S. R., Ganong, A. H., & Brown, T. H. (1986). Hebbian synapses in hippocampus. Proc. Natl. Acad. Sci. USA, 83, 5326-5330.
41. Kempter, R., Gerstner, W., & van Hemmen, J. L. (1999). Hebbian learning and spiking neurons. Phys. Rev. E, 59, 4498-4514.
42. Kempter, R., Gerstner, W., & van Hemmen, J. L. (2001). Intrinsic stabilization of output rates by spike-based Hebbian learning. Neural Computation, 13, 2709-2741.
43. Kullmann, D. M. (2003). Silent synapses: What are they telling us about long-term potentiation? Phil. Trans. R. Soc. Lond. B: Biological Sciences, 358, 727-733.
44. Laughlin, S. (1981). A simple coding procedure enhances a neuron's information capacity. Z. Naturforschung, 36, 910-912.
45. Laughlin, S., de Ruyter van Steveninck, R. R., & Anderson, J. (1998). The metabolic cost of neural information. Nature. Neuroscience, 1, 36-41.
46. Levy, W., & Baxter, R. (2002). Energy-efficient neuronal computation via quantal synaptic failures. J. Neuroscience, 22, 4746-4755.
47. Linsker, R. (1986). From basic network principles to neural architecture: Emergence of spatial-opponent cells. Proc. Natl. Acad. Sci. USA, 83, 7508-7512.
48. Linsker, R. (1989). How to generate ordered maps by maximizing the mutual information between input and output signals. Neural Computation, 1(3), 402-411.
49. Lisman, J. (1989). A mechanism for Hebb and anti-Hebb processes underlying learning and memory. Proc. Natl. Acad. Sci. USA, 86, 9574-9578.
50. Lisman, J. (2003). Long-term potentiation: Outstanding questions and attempted synthesis. Phil. Trans. R. Soc. Lond B: Biological Sciences, 358, 829-842.
51. Malenka, R. C., Kauer, J., Zucker, R., & Nicoll, R. A. (1988). Postsynaptic calcium is sufficient for potentiation of hippocampal synaptic transmission. Science, 242, 81-84.
52. Malenka, R. C., & Nicoll, R. A. (1993). NMDA-receptor-dependent plasticity: Multiple forms and mechanisms. Trends Neurosci., 16, 480-487.
53. Markram, H., Lübke, J. L, Frotscher, M., & Sakmann, B. (1997). Regulation of synaptic efficacy by coincidence of postysnaptic AP and EPSP. Science, 275, 213-215.
54. Merzenich, M., Nelson, R., Stryker, M., Cynader, M., Schoppmann, A., & Zook, J. (1984). Somatosensory cortical map changes following digit amputation in adult monkeys. J. Comparative Neurology, 224, 591-605.
55. Miller, K., Keller, J. B., & Stryker, M. P. (1989). Ocular dominance column development: Analysis and simulation. Science, 245, 605-615.
56. Miller, K. D., & MacKay, D. J. C. (1994). The role of constraints in Hebbian learning. Neural Computation, 6, 100-126.
57. Oja, E. (1982). A simplified neuron model as a principal component analyzer. J. Mathematical Biology, 15, 267-273.
58. Petersen, C., Malenka, R., Nicoll, R., & Hopfield, J. (1998). All-or-none potentiation of CA3-CA1 synapses. Proc. Natl. Acad. Sci. USA, 95, 4732-4737.
59. Pfister, J.-P., Toyoizumi, T., Barber, D., & Gerstner, W. (2006). Optimal spike-timing dependent plasticity for precise action potential firing in supervised learning. Neural Computation, 18, 1309-1339.
60. Rubin, J., Lee, D. D., & Sompolinsky, H. (2001). Equilibrium properties of temporally asymmetric Hebbian plasticity. Physical Review Letters, 86, 364-367.
61. Schultz, W., Dayan, P., & Montague, R. (1997). A neural substrate for prediction and reward. Science, 275, 1593-1599.
62. Senn, W., Tsodyks, M., & Markram, H. (2001). An algorithm for modifying neurotransmitter release probability based on pre- and postsynaptic spike timing. Neural Computation, 13, 35-67.
63. Seung, H. (2003). Learning in spiking neural networks by reinforcement of stochastic synaptic transmission. Neuron, 40, 1063-1073.
64. Sjöström, P., Turrigiano, G., & Nelson, S. (2001). Rate, timing, and cooperativity jointly determine cortical synaptic plasticity. Neuron, 32, 1149-1164.
65. Sjöström, P., Turrigiano, G., & Nelson, S. (2004). Endocannabinoid-dependent neocortical layer-5 LTD in the absence of postsynaptic spiking. J. Neurophysiol, 92, 3338-3343.
66. Song, S., & Abbott, L. (2001). Column and map development and cortical re-mapping through spike-timing dependent plasticity. Neuron, 32, 339-350.
67. Song, S., Miller, K., & Abbott, L. (2000). Competitive Hebbian learning through spike-time-dependent synaptic plasticity. Nature Neuroscience, 3, 919-926.
68. Toyoizumi, T., Pfister, J.-P, Aihara, K., & Gerstner, W. (2005a). Generalized Bienenstock-Cooper-Munro rule for spiking neurons that maximizes information transmission. Proc. National Academy Sciences (USA), 102, 5239-5244.
69. Toyoizumi, T., Pfister, J.-P., Aihara, K., & Gerstner, W. (2005b). Spike-timing dependent plasticity and mutual information maximization for a spiking neuron model. In L. K. Saul, Y. Weiss, & L. Bottou (Eds.), Advances in neural information processing systems, 17 (pp. 1409-1416). Cambridge, MA: MIT Press.
70. Turrigiano, G., & Nelson, S. (2004). Homeostatic plasticity in the developing nervous system. Nature. Reviews Neuroscience, 5, 97-107.
71. van Rossum, M. C. W., Bi, G. Q., & Turrigiano, G. G. (2000). Stable Hebbian learning from spike timing-dependent plasticity. J. Neuroscience, 20, 8812-8821.
72. von der Malsburg, C. (1973). Self-organization of orientation selective cells in the striate cortex. Kybernetik, 14,85-100.
73. Yeung, L. C., Shouval, H., Biais, B., & Cooper, L. (2004). Synaptic homeostasis and input selectivity follow from a model of calcium dependent plasticity. Proc. Nat. Acad. Sci. USA, 101, 14943-14948.
74. Zhang, L., Tao, H., Holt, C., Harris, W. A., & Poo, M.-M. (1998). A critical window for cooperation and competition among developing retinotectal synapses. Nature, 395, 37-44.
75. Zhou, Q., Tao, H., & Poo, M. (2003). Reversal and stabilization of synaptic modifications in the developing visual system. Science, 300, 1953-1957.