Transferring learning from external to internal weights in Echo-State networks with sparse connectivity

PLoS ONE

Volumn 7, Issue 5, 2012, Pages

  Sussillo, David ^a   Abbott, L F ^b  

^a Stanford University   (United States)

^b Och Spine at New York Presbyterian Hospitals   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; CONTROLLED STUDY; CYTOARCHITECTURE; ECHO STATE NETWORK; FEEDBACK SYSTEM; LEARNING ALGORITHM; MATHEMATICAL COMPUTING; NERVE CELL NETWORK; NERVE FUNCTION; NETWORK LEARNING; SIGNAL TRANSDUCTION; TASK PERFORMANCE; ALGORITHM; ARTIFICIAL NEURAL NETWORK; LEARNING;

ALGORITHMS; FEEDBACK; NEURAL NETWORKS (COMPUTER); TRANSFER (PSYCHOLOGY);

EID: 84861401423     PISSN: None     EISSN: 19326203     Source Type: Journal    
DOI: 10.1371/journal.pone.0037372     Document Type: Article

References (19)

1. Doya K, (1992) Bifurcations in the learning of recurrent neural networks. Proceedings of the IEEE International Symposium on Circuits and Systems, ISCAS '92, vol. 6: 2777-2780.
2. Bengio Y, Simard P, Frasconi P, (1994) Learning long-term dependencies with gradient descent is difficult. IEEE Transactions on Neural Networks 5: 157-166.
3. Martens J, Sutskever I, (2011) Learning recurrent neural networks with hessian-free optimization. Proceedings of the 28th International Conference on Machine Learning. 4 Available: http://www.cs.toronto.edu/~jmartens/docs/RNN_HF.pdf. Accessed 2012 May.
4. Maass W, Natschläger T, Markram H, (2002) Real-time computing without stable states: a new framework for neural computation based on perturbations. Neural Computation 14: 2531-2560.
5. Jaeger H, (2003) Adaptive nonlinear system identification with echo state networks. In: Becker S, Thrun S, Obermayer K, editors. Advances in Neural Information Processing Systems 15 Cambridge, MA MIT Press. 1713 pp.
6. Jaeger H, Haas H, (2004) Harnessing nonlinearity: predicting chaotic systems and saving energy in wireless communication. Science 304: 78-80.
7. Sussillo D, Abbott LF, (2009) Generating coherent patterns of activity from chaotic neural networks. Neuron 63: 544-557.
8. Maass W, Joshi P, Sontag ED, (2007) Computational aspects of feedback in neural circuits. PLoS Comput Biol 3: e165.
9. Jaeger H, (2010) Reservoir self-control for achieving invariance against slow input distortions. Jacobs University technical report No. 23. 4 Available: http://minds.jacobs-university.de/sites/default/files/uploads/papers/ReservoirSelfControl_Techrep.pdf. Accessed 2012 May.
10. Li J, Jaeger H, (2011) Minimal energy control of an esn pattern generator. Jacobs University technical report No. 26. 4 Available: http://minds.jacobs-university.de/sites/default/files/uploads/papers/2399_LiJaeger11.pdf. Accessed 2012 May.
11. Reinhart R, Steil J, (2011) Reservoir regularization stabilizes learning of Echo State Networks with output feedback. European Symp on ANNs: d-facto, 59-64.
12. Mayer NM, Browne M, (2004) Lecture Notes in Computer Science. In: Ijspeert AJ, Murata M, Wakamiya N, editors. Biologically Inspired Approaches to Advanced Information Technology, volume 3141 Berlin Springer pp. 40-48.
13. Buonomano DV, Merzenich MM, (1995) Temporal information transformed into a spatial code by a neural network with realistic properties. Science 267: 1028-1030.
14. Haykin S, (2001) Adaptive Filter Theory, 4th ed. Upper Saddle River, NJ Prentice Hall.
15. Sompolinsky H, Crisanti A, Sommers H, (1988) Chaos in random neural networks. Physical Review Letters 61: 259-262.
16. Abbott L, Rajan K, Sompolinsky H, (2011) Interactions between intrinsic and stimulus-dependent activity in recurrent neural networks. In: Ding M, Glanzman, D, editors. The Dynamic Brain: An Exploration of Neuronal Variability and Its Functional Significance Oxford Oxford University Press pp. 65-82.
17. Rajan K, Abbott L, Sompolinsky H, (2010) Stimulus-dependent suppression of chaos in recurrent neural networks. Physical Review E 82.
18. Candes EJ, Romberg J, Tao T, (2006) Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information. IEEE Transactions on Information Theory 52: 489-509.
19. Candes EJ, Plan Y, (2010) A Probabilistic and RIPless Theory of Compressed Sensing. IEEE Transactions on Information Theory 57: 7235-7254.