Learning in spiking neural networks by reinforcement of stochastic synaptic transmission

Neuron

Volumn 40, Issue 6, 2003, Pages 1063-1073

  Seung, H Sebastian ^a  

^a MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EVOLUTION; FACILITATION; GENE MUTATION; HYPOTHESIS; LEARNING; MATHEMATICAL COMPUTING; NERVE CELL NETWORK; NERVE CELL PLASTICITY; NERVE POTENTIAL; NEUROMODULATION; PRIORITY JOURNAL; REINFORCEMENT; REVIEW; SIGNAL TRANSDUCTION; SIMULATION; STATISTICS; STOCHASTIC MODEL; SYNAPSE VESICLE; SYNAPTIC TRANSMISSION;

EID: 0347362917     PISSN: 08966273     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0896-6273(03)00761-X     Document Type: Article

References (47)

1. Bartlett, P.L., and Baxter, J. (2000). A biologically plausible and locally optimal learning algorithm for spiking neurons. Technical Report Australian National University.
2. Barto A.G., Sutton R.S., Anderson C.W. Neuronlike adaptive elements that can solve difficult learning control problems. IEEE Trans. Syst. Man Cybern. 13:1983;834-846.
3. Baum E.B. Toward a model of intelligence as an economy of agents. Mach. Learn. 35:1999;155-185.
4. Baxter J., Bartlett P.L. Infinite-horizon policy-gradient estimation. Journal of Artificial Intelligence Research. 15:2001;319-350.
5. Bi G.Q., Poo M.M. Synaptic modifications in cultured hippocampal neurons. dependence on spike timing, synaptic strength, and postsynaptic cell type J. Neurosci. 18:1998;10464-10472.
6. Bolshakov V.Y., Siegelbaum S.A. Regulation of hippocampal transmitter release during development and long-term potentiation. Science. 269:1995;1730-1734.
7. Buchs N.J., Senn W. Spike-based synaptic plasticity and the emergence of direction selective simple cells. simulation results J. Comput. Neurosci. 13:2002;167-186.
8. Cauwenberghs G. A fast stochastic error-descent algorithm for supervised learning and optimization. Adv. Neural Info. Proc. Syst. 5:1993;244-251.
9. Chance F.S., Nelson S.B., Abbott L.F. Synaptic depression and the temporal response characteristics of v1 cells. J. Neurosci. 18:1998;4785-4799.
10. Dale N., Schacher S., Kandel E.R. Long-term facilitation in aplysia involves increase in transmitter release. Science. 239:1988;282-285.
11. Davison M., McCarthy D. The Matching Law. A Research Review:1988;Erlbaum, Hillsdale, NJ.
12. Dayan P. Reinforcement comparison. Touretzky D.S., Elman J.L., Sejnowski T.J., Hinton G.E. Proceedings of the 1990 Connectionist Models Summer School. 1990;45-51 Morgan Kaufmann, San Mateo, CA.
13. Fuhrmann G., Segev I., Markram H., Tsodyks M. Coding of temporal information by activity-dependent synapses. J. Neurophysiol. 87:2002;140-148.
14. Gallistel C.R. Foraging for brain stimulation. toward a neurobiology of computation Cognition. 50:1994;151-170.
15. Herrnstein R.J., Prelec D. Melioration. a theory of distributed choice J. Econ. Perspect. 5:1991;137-156.
16. Hinton G.E. Connectionist learning procedures. Artif. Intell. 40:1989;185-234.
17. Ito M. Cerebellar long-term depression. characterization, signal transduction, and functional roles Physiol. Rev. 81:2001;1143-1195.
18. Jabri M., Flower B. Weight perturbation - an optimal architecture and learning technique for analog VLSI feedforward and recurrent multilayer networks. IEEE Trans. Neural Netw. 3:1992;154-157.
19. Johnson F., Soderstrom K., Whitney O. Quantifying song bout production during zebra finch sensory-motor learning suggests a sensitive period for vocal practice. Behav. Brain Res. 131:2002;57-65.
20. Kimura, H., Yamamura, M., and Kobayashi, S. (1995). Reinforcement learning by stochastic hill climbing on discounted reward. In Proceedings of the 12th International Conference on Machine Learning, A. Prieditis and S. Russell, eds. (San Francisco: Morgan Kaufmann), pp. 295-303.
21. Klopf A.H. The Hedonistic Neuron. 1982;Hemisphere, Washington.
22. Kushner H.J., Clark D.S. Stochastic Approximation Methods for Constrained and Unconstrained Systems. 1978;Springer-Verlag, New York.
23. Magee J.C., Johnston D. A synaptically controlled, associative signal for Hebbian plasticity in hippocampal neurons. Science. 275:1997;209-213.
24. Mainen Z.F., Sejnowski T.J. Reliability of spike timing in neocortical neurons. Science. 268:1995;1503-1506.
25. Malenka R.C., Nicoll R.A. Long-term potentiation - a decade of progress? Science. 285:1999;1870-1874.
26. Markram H., Tsodyks M. Redistribution of synaptic efficacy between neocortical pyramidal neurons. Nature. 382:1996;807-810.
27. Markram H., Lubke J., Frotscher M., Sakmann B. Regulation of synaptic efficacy by coincidence of postsynaptic APs and EPSPs. Science. 275:1997;213-215.
28. Mazzoni P., Anderson R.A., Jordan M.I. A more biologically plausible learning rule for neural networks. Proc. Natl. Acad. Sci. USA. 88:1991;4433-4437.
29. Menzel R. Searching for the memory trace in a mini-brain, the honeybee. Learn. Mem. 8:2001;53-62.
30. Minsky, M.L. (1954). Theory of neural-analog reinforcement systems and its application to the brain-model problem. PhD thesis, Princeton University, Princeton, New Jersey.
31. Minsky M. Steps toward artificial intelligence. Proc. IRE. 49:1961;8-30.
32. Minsky M. Society of Mind. 1988;Simon & Schuster, New York.
33. Montague P.R., Dayan P., Sejnowski T.J. A framework for mesencephalic dopamine systems based on predictive hebbian learning. J. Neurosci. 16:1996;1936-1947.
34. Narendra K.S., Thathachar M.A.L. Learning Automata. An Introduction:1989;Prentice Hall, Englewood Cliffs, NJ.
35. Raymond J.L., Lisberger S.G. Neural learning rules for the vestibulo-ocular reflex. J. Neurosci. 18:1998;9112-9129.
36. Rumelhart D.E., Hinton G.E., Williams R.J. Learning internal representations by back-propagating errors. Nature. 323:1986;533-536.
37. Schultz W. Getting formal with dopamine and reward. Neuron. 36:2002;241-263.
38. Stevens C.F. Quantal release of neurotransmitter and long-term potentiation. Cell Suppl. 72:1993;55-63.
39. Stevens C.F., Wang Y. Changes in reliability of synaptic function as a mechanism for plasticity. Nature. 371:1994;704-707.
40. Sutton, R.S. (1984). Temporal credit assignment in reinforcement learning. PhD thesis, University of Massachusetts, Amherst, Amherst, Massachusetts.
41. Tesauro G. Practical issues in temporal difference learning. Mach. Learn. 8:1992;257-277.
42. Thomson A.M. Facilitation, augmentation and potentiation at central synapses. Trends Neurosci. 23:2000;305-312.
43. Tsodyks M., Pawelzik K., Markram H. Neural networks with dynamic synapses. Neural Comput. 10:1998;821-835.
44. Unnikrishnan K.P., Venugopal K.P. Alopex. a correlation based learning algorithm for feed-forward and recurrent neural networks Neural Comput. 6:1994;469-490.
45. van Vreeswijk C., Sompolinsky H. Chaotic balanced state in a model of cortical circuits. Neural Comput. 10:1998;1321-1371.
46. Werfel J.K., Xie X., Seung H.S. Learning curves for stochastic gradient descent in linear feedforward networks. Adv. Neural Info. Proc. Syst. in press:2004.
47. Williams R.J. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Mach. Learn. 8:1992;229-256.