Learning the pseudoinverse solution to network weights

Neural Networks

Volumn 45, Issue , 2013, Pages 94-100

  Tapson, J ^a   van Schaik, A ^a  

^a WESTERN SYDNEY UNIVERSITY   (Australia)

Author keywords

Biological plausibility; Extreme learning machine; Moore Penrose pseudoinverse; Neural engineering

Indexed keywords

BIOLOGICAL PLAUSIBILITY; BIOLOGICALLY PLAUSIBLE NEURAL NETWORKS; COMPUTATIONAL NEUROSCIENCE; EXTREME LEARNING MACHINE; NEURAL ENGINEERING; NEURAL NETWORK STRUCTURES; NEUROMORPHIC ENGINEERING; PSEUDO-INVERSES;

LEARNING SYSTEMS; SINGULAR VALUE DECOMPOSITION;

NEURAL NETWORKS;

ARTICLE; CONTROLLED STUDY; INTERMETHOD COMPARISON; LEARNING ALGORITHM; LINEAR SYSTEM; MACHINE LEARNING; MATHEMATICAL ANALYSIS; MATHEMATICAL COMPUTING; ONLINE ANALYSIS; PRIORITY JOURNAL; PSEUDOINVERSE; STATISTICAL CONCEPTS;

BIOLOGICAL PLAUSIBILITY; EXTREME LEARNING MACHINE; MOORE-PENROSE PSEUDOINVERSE; NEURAL ENGINEERING;

BRAIN; BRAIN WAVES; HUMANS; LEARNING; MATHEMATICS; NEURAL NETWORKS (COMPUTER);

EID: 84880787590     PISSN: 08936080     EISSN: 18792782     Source Type: Journal    
DOI: 10.1016/j.neunet.2013.02.008     Document Type: Article

References (28)

1. Barak O., Rigotti M. A simple derivation of a bound on the perceptron margin using singular value decomposition. Neural Computation 2011, 23:1935-1943.
2. Ben-Israel A., Greville T.N.E. Generalized inverses: theory and applications 2003, Springer, New York. 2nd ed.
3. Bishop C.M. Neural networks for pattern recognition 2008, Oxford University Press, Oxford.
4. Cho Y., Saul L.K. Large-margin classification in infinite neural networks. Neural Computation 2010, 22:2678-2697.
5. Choudhary, S., Sloan, S., Fok, S., Neckar, A., Trautmann, E., & Gao, P. et al. (2012). Silicon neurons that compute. In Int. conf. artificial neural networks, ICANN 2012. Lecture Notes in Computer Science, 7552. (pp. 121-128).
6. Eliasmith C., Anderson C.H. Neural engineering: computation, representation and dynamics in neurobiological systems 2003, MIT Press.
7. Galluppi, F., Davies, S., Furber, S., Stewart, T., & Eliasmith, C. (2012). Real time on-chip implementation of dynamical systems with spiking neurons. In Proc. int. joint conf. neural networks, IJCNN 2012. (pp. 1-8).
8. Greville T.N.E. Some applications of the pseudoinverse of a matrix. SIAM Review 1960, 2:15-22.
9. Guo P., Lyu M.R. A pseudoinverse learning algorithm for feedforward neural networks with stacked generalization applications to software reliability growth data. Neurocomputing 2003, 56:101-121.
10. Huang, G. -B. (2012). Extreme learning machine: learning without iterative tuning. In Tutorial presentation at the World congress on computational intelligence, WCCI 2012 (p. 96). Available at: (on 05.12.12). http://www.ntu.edu.sg/home/egbhuang/.
11. Huang, G. -B., Liang, N. -Y., Rong, H. -J., Saratchandran, P., & Sundararajan, N. (2005). On-line sequential extreme learning machine. In Proc. IASTED international conference on computational intelligence, CI 2005.
12. Huang G.-B., Zhu Q.-Y., Siew C.-K. Extreme learning machine: theory and applications. Neurocomputing 2006, 70:489-501.
13. Kohonen T. Correlation matrix memories. IEEE Transactions on Computers 1988, C-21(4):353-359.
14. Kohonen T. Self-organization and associative memory 1989, Springer-Verlag, Berlin, (Chapter 6). 3rd ed.
15. Kohonen T. Self-organizing maps 2000, Springer-Verlag, Berlin, (Chapter 2). 3rd ed.
16. Kovanic P. On the pseudoinverse of a sum of symmetric matrices with applications to estimation. Kybernetika 1979, 15(5):341-348.
17. LeCun Y., Bottou L., Bengio Y., Haffner P. Gradient-based learning applied to document recognition. Proceedings of the IEEE 1998, 86(11):2278-2324.
18. LeCun, Y., & Cortes, C. (2012). The MNIST database of handwritten digits. Available at: (last accessed: 12.07.12). http://yann.lecun.com/exdb/mnist/.
19. Mackey M., Glass L. Oscillation and chaos in physiological control systems. Science 1977, 197:287.
20. MacNeil D., Eliasmith C. Fine-tuning and the stability of recurrent neural networks. PLoS One 2011, 6(9):e22885.
21. Oja E. Neural networks, principal components, and subspaces. International Journal of Neural Systems 1989, 1:61-68.
22. O'Reilly R.C. Six principles for biologically-based computational models of cortical cognition. Trends in Cognitive Sciences 1998, 2:455-462.
23. Penrose Roger A generalized inverse for matrices. Proceedings of the Cambridge Philosophical Society 1955, 51:406-413.
24. Rahimi A., Recht B. Weighted sums of random kitchen sinks: replacing minimization with randomization in learning. Advances in neural information processing systems, 21 2009, 1313-1320. MIT Press, Cambridge, MA. D. Koller, D. Schuurmans, Y. Bengio, L. Bottou (Eds.).
25. Rigotti M., Ben Dayan Rubin D., Wang X.-J., Fusi S. Internal representation of task rules by recurrent dynamics: the importance of the diversity of neural responses. Frontiers in Computational Neuroscience 2010, 4:24. 10.3389/fncom.2010.00024.
26. Saxe, A., Koh, P., Chen, Z., Bhand, M., Suresh, B., & Ng, A. (2011). On random weights and unsupervised feature learning. In Proc. 28th int. conf. on machine learning.
27. Schwartz O., Simoncelli E. Natural signal statistics and sensory gain control. Nature Neuroscience 2001, 4:819-825.
28. Stewart T.C., Bekolay T., Eliasmith C. Learning to select actions with spiking neurons in the basal ganglia. Frontiers in Decision Neuroscience 2012, 6. 10.3389/fnins.2012.00002.