Spiking neurons can discover predictive features by aggregate-label learning

Science

Volumn 351, Issue 6277, 2016, Pages

  Gütig, Robert ^a  

^a MAX PLANCK INSTITUTE OF EXPERIMENTAL MEDICINE   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

BRAIN; CELLS AND CELL COMPONENTS; INFORMATION; LEARNING; SENSORY SYSTEM;

AGGREGATE LABEL LEARNING; ARTICLE; FEEDBACK SYSTEM; MACHINE LEARNING; NERVE CELL; NERVE CELL MEMBRANE POTENTIAL; NERVE CELL NETWORK; NERVE POTENTIAL; NEUROLOGIC EXAMINATION; PRIORITY JOURNAL; REINFORCEMENT; REWARD; SENSORY SYSTEM; SPEECH DISCRIMINATION; SPEECH RECOGNITION TASK; SPIKE WAVE; SYNAPSE; ARTIFICIAL NEURAL NETWORK; BIOLOGICAL MODEL; BRAIN; CONCEPT FORMATION; CONSTRUCTIVE FEEDBACK; HUMAN; LEARNING; MEMBRANE POTENTIAL; PHYSIOLOGY; SPEECH; SUPERVISED MACHINE LEARNING; UNSUPERVISED MACHINE LEARNING;

BRAIN; CONCEPT FORMATION; FORMATIVE FEEDBACK; HUMANS; LEARNING; MEMBRANE POTENTIALS; MODELS, NEUROLOGICAL; NEURAL NETWORKS (COMPUTER); NEURONS; REINFORCEMENT (PSYCHOLOGY); SPEECH; SUPERVISED MACHINE LEARNING; UNSUPERVISED MACHINE LEARNING;

EID: 84961221485     PISSN: 00368075     EISSN: 10959203     Source Type: Journal    
DOI: 10.1126/science.aab4113     Document Type: Article

References (47)

1. R. S. Sutton, Learning to predict by the methods of temporal differences. Mach. Learn. 3, 9 (1988).
2. W. Schultz, P. Dayan, P. R. Montague, A neural substrate of prediction and reward. Science 275, 1593-1599 (1997). doi: 10.1126/science.275.5306.1593; pmid: 9054347
3. E. M. Izhikevich, Solving the distal reward problem through linkage of STDP and dopamine signaling. Cereb. Cortex 17, 2443-2452 (2007). doi: 10.1093/cercor/bhl152; pmid: 17220510
4. J. Friedrich, R. Urbanczik, W. Senn, Spatio-temporal credit assignment in neuronal population learning. PLOS Comput. Biol. 7, e1002092 (2011). doi: 10.1371/journal.pcbi.1002092; pmid: 21738460
5. F. Ponulak, A. Kasiński, Supervised learning in spiking neural networks with ReSuMe: Sequence learning, classification, and spike shifting. Neural Comput. 22, 467-510 (2010). doi: 10.1162/neco.2009.11-08-901; pmid: 19842989
6. R. V. Florian, The chronotron: A neuron that learns to fire temporally precise spike patterns. PLOS ONE 7, e40233 (2012). doi: 10.1371/journal.pone.0040233; pmid: 22879876
7. Y. Xu, X. Zeng, S. Zhong, A new supervised learning algorithm for spiking neurons. Neural Comput. 25, 1472-1511 (2013). doi: 10.1162/NECO-a-00450; pmid: 23517101
8. R. M. Memmesheimer, R. Rubin, B. P. Olveczky, H. Sompolinsky, Learning precisely timed spikes. Neuron 82, 925-938 (2014). doi: 10.1016/j.neuron.2014.03.026; pmid: 24768299
9. F. Rosenblatt, Principles of Neurodynamics: Perceptrons and the Theory of Brain Mechanisms (Spartan Books, 1962).
10. R. Gütig, To spike, or when to spike? Curr. Opin. Neurobiol. 25, 134-139 (2014). doi: 10.1016/j.conb.2014.01.004; pmid: 24468508
11. H. S. Seung, Learning in spiking neural networks by reinforcement of stochastic synaptic transmission. Neuron 40, 1063-1073 (2003). doi: 10.1016/S0896-6273(03)00761-X; pmid: 14687542
12. R. Gütig, H. Sompolinsky, The tempotron: A neuron that learns spike timing-based decisions. Nat. Neurosci. 9, 420-428 (2006). doi: 10.1038/nn1643; pmid: 16474393
13. I. R. Fiete, M. S. Fee, H. S. Seung, Model of birdsong learning based on gradient estimation by dynamic perturbation of neural conductances. J. Neurophysiol. 98, 2038-2057 (2007). doi: 10.1152/jn.01311.2006; pmid: 17652414
14. L. R. Rabiner, A tutorial on hidden Markov models and selected applications in speech recognition. Proc. IEEE 77, 257-286 (1989). doi: 10.1109/5.18626
15. At first sight, it seems that losing the clue timing eliminates almost all the necessary feedback information, and asking for a specific number of spikes seems irrelevant because we do not really care about the total number of spikes if we only get at least one spike for each clue. However, a varying spike count per clue would indicate that the detector is operating in a regime in which noise could suppress or add spikes, which implies that the detector would not function reliably when the clues are embedded in background activity. This suggests that the number of spikes could serve as an objective function to stabilize an effective synaptic configuration and might also suffice for learning such a configuration from random initial conditions.
16. Because a neuron's memory for previous inputs decays rapidly (neural membrane time constants rarely exceed a few tens of milliseconds), it has to fire rather promptly in response to a clue, if at all.
17. B. Widrow, M. A. Lehr, 30 years of adaptive neural networks: Perceptron, Madaline, and backpropagation. Proc. IEEE 78, 1415-1442 (1990). doi: 10.1109/5.58323
18. D. Barber, "Learning in spiking neural assemblies," in Advances in Neural Information Processing Systems, S. Becker, S. Thrun, K. Obermayer, Eds. (MIT Press, 2002), vol. 15, pp. 149-156.
19. J. P. Pfister, T. Toyoizumi, D. Barber, W. Gerstner, Optimal spike-timing-dependent plasticity for precise action potential firing in supervised learning. Neural Comput. 18, 1318-1348 (2006). doi: 10.1162/neco.2006.18.6.1318; pmid: 16764506
20. Although a population of 135 neurons (with 80 synapses each) required more than 3000 trials of reinforcement learning (4), the single integrate-and-fire neuron underlying the present study required an average of eight or four trials, when learning was implemented with the multi-spike tempotron or the correlation-based approximation (discussion and materials and methods, correlation-based learning), respectively. In fact, based on aggregate labels the task could be readily solved by a single neuron with only 20 synapses (table S2). Similarly strong performance differences between other neural implementations of gradient-based and reinforcement learning (11) have been reported (12) in the context of a binary classification task.
21. T. Yang, M. N. Shadlen, Probabilistic reasoning by neurons. Nature 447, 1075-1080 (2007). doi: 10.1038/nature05852; pmid: 17546027
22. L. F. Abbott, S. B. Nelson, Synaptic plasticity: Taming the beast. Nat. Neurosci. 3 (suppl.), 1178-1183 (2000). doi: 10.1038/81453; pmid: 11127835
23. R. Gütig, H. Sompolinsky, Time-warp-invariant neuronal processing. PLOS Biol. 7, e1000141 (2009). doi: 10.1371/ journal.pbio.1000141; pmid: 19582146
24. R. G. Leonard, G. Doddington, TIDIGITS, vol. LDC93S10 (Linguistic Data Consortium, 1993).
25. T. Masquelier, R. Guyonneau, S. J. Thorpe, Spike timing dependent plasticity finds the start of repeating patterns in continuous spike trains. PLOS ONE 3, e1377 (2008). doi: 10.1371/journal.pone.0001377; pmid: 18167538
26. B. Nessler, M. Pfeiffer, L. Buesing, W. Maass, Bayesian computation emerges in generic cortical microcircuits through spike-timing-dependent plasticity. PLOS Comput. Biol. 9, e1003037 (2013). pmid: 23633941
27. R. Linsker, Perceptual neural organization: Some approaches based on network models and information theory. Annu. Rev. Neurosci. 13, 257-281 (1990). doi: 10.1146/ annurev.ne.13.030190.001353; pmid: 2183677
28. R. Gütig, R. Aharonov, S. Rotter, H. Sompolinsky, Learning input correlations through nonlinear temporally asymmetric Hebbian plasticity. J. Neurosci. 23, 3697-3714 (2003). pmid: 12736341
29. W. Schultz, Updating dopamine reward signals. Curr. Opin. Neurobiol. 23, 229-238 (2013). doi: 10.1016/ j.conb.2012.11.012; pmid: 23267662
30. V. Pawlak, J. R. Wickens, A. Kirkwood, J. N. Kerr, Timing is not everything: Neuromodulation opens the STDP gate. Front. Synaptic Neurosci. 2, 146 (2010). doi: 10.3389/ fnsyn.2010.00146; pmid: 21423532
31. A. Artola, S. Bröcher, W. Singer, Different voltage-dependent thresholds for inducing long-term depression and long-term potentiation in slices of rat visual cortex. Nature 347, 69-72 (1990). doi: 10.1038/347069a0; pmid: 1975639
32. D. E. Feldman, The spike-timing dependence of plasticity. Neuron 75, 556-571 (2012). doi: 10.1016/j.neuron.2012.08.001; pmid: 22920249
33. V. Pawlak, D. S. Greenberg, H. Sprekeler, W. Gerstner, J. N. Kerr, Changing the responses of cortical neurons from sub- to suprathreshold using single spikes in vivo. eLife 2, e00012 (2013). doi: 10.7554/eLife.00012; pmid: 23359858
34. J. Amores, Multiple instance classification: Review, taxonomy and comparative study. Artif. Intell. 201, 81-105 (2013). doi: 10.1016/j.artint.2013.06.003
35. H. Hotelling, Relations between two sets of variates. Biometrika 28, 321-377 (1936). doi: 10.1093/biomet/28.3-4.321
36. S. Becker, G. E. Hinton, Self-organizing neural network that discovers surfaces in random-dot stereograms. Nature 355, 161-163 (1992). doi: 10.1038/355161a0; pmid: 1729650
37. S. Becker, Mutual information maximization: models of cortical self-organization. Network 7, 7-31 (1996). doi: 10.1088/ 0954-898X/7/1/003
38. Y. Li, S. D. Van Hooser, M. Mazurek, L. E. White, D. Fitzpatrick, Experience with moving visual stimuli drives the early development of cortical direction selectivity. Nature 456, 952-956 (2008). doi: 10.1038/nature07417; pmid: 18946471
39. S. D. Van Hooser et al., Initial neighborhood biases and the quality of motion stimulation jointly influence the rapid emergence of direction preference in visual cortex. J. Neurosci. 32, 7258-7266 (2012). doi: 10.1523/JNEUROSCI.0230-12.2012; pmid: 22623671
40. L. Wiskott, T. J. Sejnowski, Slow feature analysis: Unsupervised learning of invariances. Neural Comput. 14, 715-770 (2002). doi: 10.1162/089976602317318938; pmid: 11936959
41. N. Li, J. J. DiCarlo, Unsupervised natural experience rapidly alters invariant object representation in visual cortex. Science 321, 1502-1507 (2008). pmid: 18787171
42. R. Min, M. Santello, T. Nevian, The computational power of astrocyte mediated synaptic plasticity. Front. Comput. Neurosci. 6, 93 (2012). doi: 10.3389/fncom.2012.00093; pmid: 23125832
43. P. Poirazi, T. Brannon, B. W. Mel, Pyramidal neuron as twolayer neural network. Neuron 37, 989-999 (2003). doi: 10.1016/S0896-6273(03)00149-1; pmid: 12670427
44. M. Larkum, A cellular mechanism for cortical associations: An organizing principle for the cerebral cortex. Trends Neurosci. 36, 141-151 (2013). doi: 10.1016/j.tins.2012.11.006; pmid: 23273272
45. J. A. Cummings, R. M. Mulkey, R. A. Nicoll, R. C. Malenka, Ca2+ signaling requirements for long-term depression in the hippocampus. Neuron 16, 825-833 (1996). doi: 10.1016/ S0896-6273(00)80102-6; pmid: 8608000
46. R. C. Malenka, R. A. Nicoll, Long-term potentiation-A decade of progress? Science 285, 1870-1874 (1999). doi: 10.1126/ science.285.5435.1870; pmid: 10489359
47. J. Hertz, A. Krough, R. G. Palmer, Introduction to the Theory of Neural Computation (Westview Press, 1991).