Stochastic variational learning in recurrent spiking networks

Frontiers in Computational Neuroscience

Volumn 8, Issue 1 APR, 2014, Pages

  Jimenez Rezende, Danilo ^a   Gerstner, Wulfram ^a  

^a EPFL   (Switzerland)

Author keywords

Action potentials; Neural networks; Spiking neurons; Synapses; Variational learning

Indexed keywords

COMPUTER SIMULATION; ELECTROPHYSIOLOGY; NEURONS; REINFORCEMENT LEARNING;

ACTION POTENTIALS; RECURRENT NETWORKS; SPIKE TIMING DEPENDENT PLASTICITIES; SPIKING NETWORKS; SPIKING NEURON; STATISTICAL INFERENCE; SYNAPSES; VARIATIONAL LEARNING;

NEURAL NETWORKS;

ARTICLE; INTERMETHOD COMPARISON; LEARNING; MATHEMATICAL MODEL; NERVE CELL NETWORK; NERVE CELL PLASTICITY; NEUROMODULATION; REINFORCEMENT LEARNING; SIMULATION; STATISTICS; VARIATIONAL LEARNING;

EID: 84897942397     PISSN: 16625188     EISSN: None     Source Type: Journal    
DOI: 10.3389/fncom.2014.00038     Document Type: Article

References (54)

1. Artola, A., and Singer, W. (1993). Long-term depression of excitatory synaptic transmission and its relationship to long-term potentiation. Trends Neurosci. 16, 480-487. doi: 10.1016/0166-2236(93)90081-V
2. Beal, M. J., and Ghahramani, Z. (2006). Variational bayesian learning of directed graphical models with hidden variables. Bayesian Anal. 1, 793-832. doi: 10.1214/06-BA126
3. Bengio, Y., Simard, P., and Frasconi, P. (1994). Learning long-term dependencies with gradient descent is difficult. IEEE Trans. Neural Netw. 5, 157-166. doi: 10.1109/72.279181
4. Berkes, P., Orban, G., Lengyel, M., and Fiser, J. (2011). Spontaneous cortical activity reveals hallmarks of an optimal internal model of the environment. Science 331, 83-87. doi: 10.1126/science.1195870
5. Bhatnagar, S., Sutton, R., Ghavamzadeh, M., and Lee, M. (2007). Incremental natural actor-critic algorithms. Adv. Neural Inf. Process. Syst. 20, 105-112.
6. Bi, G., and Poo, M. (2001). Synaptic modification of correlated activity: Hebb's postulate revisited. Annu. Rev. Neurosci. 24, 139-166. doi: 10.1146/annurev.neuro.24.1.139
7. Brea, J., Senn, W., and Pfister, J. (2011). Sequence learning with hidden units in spiking neural networks. Adv. Neural Inf. Process. Syst. 24, 1422-1430.
8. Brea, J., Senn, W., and Pfister, J.-P. (2013). Matching recall and storage in sequence learning with spiking neural networks. J. Neurosci. 33, 9565-9575. doi: 10.1523/JNEUROSCI.4098-12.2013
9. Cateau, H., Kitano, K., and Fukai, T. (2008). Interplay between a phase respone curve and spike-timing dependent plasticity leading to wireless clustering. Phys. Rev. E 77:051909. doi: 10.1103/PhysRevE.77.051909
10. Clopath, C., Busing, L., Vasilaki, E., and Gerstner, W. (2010). Connectivity reflects coding: a model of voltage-based spike-timing-dependent-plasticity with home-ostasis. Nat. Neurosci. 13, 344-352. doi: 10.1038/nn.2479
11. Dayan, P. (2000). "Helmholtz machines and wake-sleep learning," in Handbook of Brain Theory and Neural Network, ed M. A. Arbib (Cambridge, MA: MIT Press), 44.
12. Deneve, S. (2008). Bayesian spiking neurons i: inference. Neural Comput. 20, 91-117. doi: 10.1162/neco.2008.20.1.91
13. Fremaux, N., Sprekeler, H., and Gerstner, W. (2010). Functional requirements for reward-modulated spike-timing-dependent plasticity. J. Neurosci. 40, 13326-13337. doi: 10.1523/JNEUROSCI.6249-09.2010
14. Friston, K. J., and Stephan, K. E. (2007). Free-energy and the brain. Synthese 159, 417-458. doi: 10.1007/s11229-007-9237-y
15. Gerstner, W., Kempter, R., van Hemmen, J. L., and Wagner, H. (1996). A neu-ronal learning rule for sub-millisecond temporal coding. Nature 383, 76-78. doi: 10.1038/383076a0
16. Gerstner, W., and Kistler, W. K. (2002). Spiking Neuron Models. Single Neurons, Populations, Plasticity. Cambridge: Cambridge University Press. doi: 10.1017/CBO9780511815706
17. Gibbs, A. L., and Su, F. E. (2002). On choosing and bounding probability metrics. Int. Stat. Rev. 70, 419. doi: 10.2307/1403865
18. Gilson, M., Burkitt, A., Grayden, D., Thomas, D., and van Hemmen, J. L. (2009). Emergence of network structure due to spike-timing-dependent plasticity in recurrent neuronal networks iv: structuring synaptic pathways among recurrent connections. Biol. Cybern. 27, 427-444. doi: 10.1007/s00422-009-0346-1
19. Gu, Q. (2002). Neuromodulatory transmitter systems in the cortex and their role in cortical plasticity. Neuroscience 111, 815-835. doi: 10.1016/S0306-4522(02)00026-X
20. Guyonneau, R., VanRullen, R., and Thorpe, S. (2005). Neurons tune to the earliest spikes through stdp. Neural Comput. 17, 859-879. doi: 10.1162/0899766053429390
21. Habenschuss, S., Bill, J., and Nessler, B. (2012). Homeostatic plasticity in bayesian spiking networks as expectation maximization with posterior constraints. Adv. Neural Inf. Process. Syst. 25, 782-790.
22. Izhikevich, E., Gally, J., and Edelman, G. (2004). Spike-timing dynamics of neu-ronal groups. Cereb. Cortex 14, 933-944. doi: 10.1093/cercor/bhh053
23. Izhikevich, E. M. (2007). Solving the distal reward problem through linkage of stdp and dopamine signaling. Cereb. Cortex 17, 2443-2452. doi: 10.1093/cercor/bhl152
24. Jaakkola, T., and Jordan, M. (2000). Bayesian parameter estimation via variational methods. Stat. Comput. 10, 25-37. doi: 10.1023/A:1008932416310
25. Jimenez Rezende, D., Wierstra, D., and Gerstner, W. (2011). Variational learning for recurrent spiking networks. Adv. Neural Inf. Process. Syst. 24, 136-144.
26. Jolivet, R., Rauch, A., Lüscher, H.-R., and Gerstner, W. (2006). Predicting spike timing of neocortical pyramidal neurons by simple threshold models. J. Comput. Neurosci. 21, 35-49. doi: 10.1007/s10827-006-7074-5
27. Kempter, R., Gerstner, W., and van Hemmen, J. L. (1999). Hebbian learning and spiking neurons. Phys. Rev. E 59, 4498-4514. doi: 10.1103/PhysRevE. 59.4498
28. Knill, D., and Pouget, A. (2004). The bayesian brain: the role of uncertainty in neural codding and computation. Tr e nd s Neu ro s c i. 27, 712-719. doi: 10.1016/j.tins.2004.10.007
29. Körding, K., and Wolpert, D. (2004). Bayesian integration in sensorimotor learning. Nature 427, 244-247. doi: 10.1038/nature02169
30. Landau, L. D., and Lifshitz, E. (1980). Statistical Physics. Elsevier Science and Technology Books.
31. LeCun, Y., and Cortes, C. (2010). MNIST Handwritten Digit Database. Available online at: http://yann.lecun.com/exdb/mnist/
32. Levy, N., Horn, D., Meilijson, I., and Ruppin, E. (2001). Distributed synchrony in a cell assembly of spiking neurons. Neural Netw. 14, 815-824. doi: 10.1016/S0893-6080(01)00044-2
33. Markram, H., Lübke, J., Frotscher, M., and Sakmann, B. (1997). Regulation of synaptic efficacy by coincidence of postysnaptic AP and EPSP. Science 275, 213-215. doi: 10.1126/science.275.5297.213
34. Miller, K., Keller, J. B., and Stryker, M. P. (1989). Ocular dominance column development: analysis and simulation. Science 245, 605-615. doi: 10.1126/sci-ence.2762813
35. Morrison, A., Aertsen, A., and Diesmann, M. (2007). Spike-timing dependent plasticity in balanced random networks. Neural Comput. 19, 1437-1467. doi: 10.1162/neco.2007.19.6.1437
36. Munos, R. (2005). "Geometric variance reduction in Markov chains: application to value function and gradient estimation," in AAAI'05 Proceedings of the 20th National Conference on Artificial Intelligence, Vol. 2, (Pittsburgh, PA: AAAI Press), 1012-1017. Available online at: http://dl.acm.org/citation.cfm?id=1619410.1619495
37. Nessler, B., Pfeiffer, M., Buesing, L., and Maass, W. (2013). Bayesian computation emerges in generic cortical microcircuits through spike-timing-dependent plasticity. PLoS Comput. Biol. 9:e1003037. doi: 10.1371/journal.pcbi. 1003037
38. Nessler, B., Pfeiffer, M., and Maass, W. (2009). Stdp enables spiking neurons to detect hidden causes of their inputs. Response 22, 1-9.
39. Ngezahayo, A., Schachner, M., and Artola, A. (2000). Synaptic activation modulates the induction of bidirectional synaptic changes in adult mouse hippocampus. J. Neurosci. 20, 2451-2458. Available online at: http://www.biomedsearch.com/nih/Synaptic-activity-modulates-induction-bidirectional/10729325.html
40. Paninski, L. (2004). Maximum likelihood estimation of cascade point-process neural encoding models. Network 15, 243-262. doi: 10.1088/0954-898X/15/4/002
41. Pecevski, D., Buesing, L., and Maass, W. (2011). Probabilistic inference in general graphical models through sampling in stochastic networks of spiking neurons. PLOS Comput. Biol. 7:e1002294. doi: 10.1371/journal.pcbi.1002294
42. Pfister, J., Toyoizumi, T., Barber, D., and Gerstner, W. (2006). Optimal spike-timing-dependent plasticity for precise action potential firing in supervised learning. Neural Comput. 18, 1318-1348. doi: 10.1162/neco.2006.18.6.1318
43. Pillow, J., Paninski, L., and Simoncelli, E. (2004). Maximum likelihood estimation of a stochastic integrate-and-fire model. Adv. Neural Inf. Process. Syst. 16, 1311-1318.
44. Ranganath, C., and Rainer, G. (2003). Neural mechanisms for detecting and remembering novel events. Nat. Rev. Neurosci. 4, 193-202. doi: 10.1038/nrn1052
45. Salakhutdinov, R., and Hinton, G. E. (2009). "Deep Boltzmann machines," in Proceedings of the Twelfth International Conference on Artificial Intelligence and Statistics (AISTATS), Vol. 5, eds David A. Van Dyk and M. Welling (Clearwater Beach, FL), 448-455. Available online at: http://www.jmlr.org/proceedings/papers/v5/salakhutdinov09a.html
46. Savin, C., Joshi, P., and Triesch, J. (2010). Independent component analysis in spiking neurons. PLOS Comput. Biol. 6:e1000757. doi: 10.1371/jour-nal.pcbi.1000757
47. Schultz, W. (2008). A neural substrate of prediction and reward. Science 275, 1593-1599. doi: 10.1126/science.275.5306.1593
48. Shao, L. Y. (2012). Linear-nonlinear-poisson neurons can do inference on deep boltzmann machines. CoRR. Arxiv.org/abs/1210.8442.
49. Sjöström, P., Turrigiano, G., and Nelson, S. (2001). Rate, timing, and coopera-tivity jointly determine cortical synaptic plasticity. Neuron 32, 1149-1164. doi: 10.1016/S0896-6273(01)00542-6
50. Song, S., and Abbott, L. (2001). Column and map development and cortical remapping through spike-timing dependent plasticity. Neuron 32, 339-350. doi: 10.1016/S0896-6273(01)00451-2
51. Song, S., Miller, K., and Abbott, L. (2000). Competitive Hebbian learning through spike-time-dependent synaptic plasticity. Nat. Neurosci. 3, 919-926. doi: 10.1038/78829
52. Sutskever, I., Hinton, G. E., and Taylor, G. W. (2008). The recurrent temporal restricted boltzmann machine. Adv. Neural Inf. Process. Syst. 21, 1601-1608.
53. Young, J., Waleszczyk, W., Wang, C., Calford, M., Dreher, B., and Obermayer, K. (2007). Cortical reorganization consistent with spike timing-but not correlation-dependent plasticity. Nat. Neurosci. 10, 887-895. doi: 10.1038/nn0807-1073c
54. Yu, A. J., and Dayan, P. (2005). Uncertainty, neuromodulation, and attention. Neuron 46, 681-692. doi: 10.1016/j.neuron.2005.04.026