Unsupervised learning of an efficient short-term memory network

Advances in Neural Information Processing Systems

Volumn 4, Issue January, 2014, Pages 3653-3661

  Vertechi, Pietro ^a   Brendel, Wieland ^a,b   Machens, Christian K ^a  

^a CHAMPALIMAUD CENTRE FOR THE UNKNOWN   (Portugal)

^b UNIVERSITY OF TÜBINGEN   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

COST FUNCTIONS; INFORMATION SCIENCE; LINEAR NETWORKS; TIME DELAY; UNSUPERVISED LEARNING;

FEEDFORWARD INPUT; FEEDFORWARD WEIGHT; KEEP TRACK OF; LEARNING RULES; LOCAL LEARNING; RECURRENT NETWORKS; SHORT TERM MEMORY; SPIKING NETWORKS;

RECURRENT NEURAL NETWORKS;

EID: 84937944034     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (23)

1. D. G. Barrett, S. Denève, and C. K. Machens. "Firing rate predictions in optimal balanced networks". In: Advances in Neural Information Processing Systems 26. 2013, pp. 1538-1546.
2. A. J. Bell and T. J. Sejnowski. "An information-maximization approach to blind separation and blind deconvolution". In: Neural comp. 7 (1995), pp. 1129-1159.
3. M. Boerlin, C. K. Machens, and S. Denève. "Predictive coding of dynamical variables in balanced spiking networks". In: PLoS Computational Biology 9.11 (2013), e1003258.
4. R. Bourdoukan et al. "Learning optimal spike-based representations". In: Advances in Neural Information Processing Systems 25. MIT Press, 2012, epub.
5. S. Fusi, P. J. Drew, and L. F. Abbott. "Cascade models of synaptically stored memories". In: Neuron 45.4 (2005), pp. 599-611.
6. S. Hochreiter and J. Schmidhuber. "Long short-term memory". In: Neural computation 9.8 (1997), pp. 1735-1780.
7. J. J. Hopfield. "Neural networks and physical systems with emergent collective computational abilities". In: Proceedings of the national academy of sciences 79.8 (1982), pp. 2554-2558.
8. J. J. Hopfield. "Neurons with graded response have collective computational properties like those of two-state neurons". In: Proc. Natl. Acad. Sci. USA 81 (1984), pp. 3088-3092.
9. T. Hu, A. Genkin, and D. B. Chklovskii. "A network of spiking neurons for computing sparse representations in an energy-efficient way". In: Neural computation 24.11 (2012), pp. 2852-2872.
10. H. Jaeger. "The "echo state" approach to analysing and training recurrent neural networks." In: German National Research Center for Information Technology. Vol. 48. 2001.
11. A. Lazaar, G. Pipa, and J. Triesch. "SORN: a self-organizing recurrent neural network". In: Frontiers in computational neuroscience 3 (2009), p. 23.
12. M. Lukoševičius and H. Jaeger. "Reservoir computing approaches to recurrent neural network training". In: Computer Science Review 3.3 (2009), pp. 127-149.
13. W. Maass, T. Natschläger, and H. Markram. "Real-time computing without stable states: A new framework for neural computation based on perturbations". In: Neural computation 14.11 (2002), pp. 2531-2560.
14. C. K. Machens, R. Romo, and C. D. Brody. "Flexible control of mutual inhibition: A neural model of two-interval discrimination". In: Science 307 (2005), pp. 1121-1124.
15. G. Major and D. Tank. "Persistent neural activity: prevalence and mechanisms". In: Curr. Opin. Neurobiol. 14 (2004), pp. 675-684.
16. B. A. Olshausen and D. J. Field. "Sparse coding with an overcomplete basis set: A strategy employed by V1?" In: Vision Research 37.23 (1997), pp. 3311-3325.
17. R. P. N. Rao and D. H. Ballard. "Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects". In: Nature neuroscience 2.1 (1999), pp. 79-87.
18. A. Renart, N. Brunel, and X.-J. Wang. "Mean-field theory of irregularly spiking neuronal populations and working memory in recurrent cortical networks". In: Computational neuroscience: A comprehensive approach (2004), pp. 431-490.
19. C. J. Rozell et al. "Sparse coding via thresholding and local competition in neural circuits". In: Neural computation 20.10 (2008), pp. 2526-2563.
20. E. P. Simoncelli and B. A. Olshausen. "Natural image statistics and neural representation". In: Ann. Rev. Neurosci. 24 (2001), pp. 1193-1216.
21. X.-J. Wang. "Probabilistic decision making by slow reverberation in cortical circuits". In: Neuron 36.5 (2002), pp. 955-968.
22. P. J. Werbos. "Backpropagation through time: what it does and how to do it". In: Proceedings of the IEEE 78.10 (1990), pp. 1550-1560.
23. J. Zylberberg, J. T. Murphy, and M. R. DeWeese. "A sparse coding model with synaptically local plasticity and spiking neurons can account for the diverse shapes of V1 simple cell receptive fields". In: PLoS Computational Biology 7.10 (2011), e1002250.