Coupled singular value decomposition of a cross-covariance matrix

International Journal of Neural Systems

Volumn 20, Issue 4, 2010, Pages 293-318

  Kaiser, Alexander ^a   Schenck, Wolfram ^a   Möller, Ralf ^a  

^a BIELEFELD UNIVERSITY   (Germany)

Author keywords

coupled learning rule; cross covariance matrix; Hebbian learning; Newton's method; Singular value decomposition

Indexed keywords

CROSS-COVARIANCE MATRIX; HEBBIAN LEARNING; LEARNING RULES; NEWTON'S METHODS; SINGULAR VALUES;

COVARIANCE MATRIX; NEWTON-RAPHSON METHOD;

SINGULAR VALUE DECOMPOSITION;

ALGORITHM; ARTICLE; LEARNING; MATHEMATICS; PRINCIPAL COMPONENT ANALYSIS; STATISTICS;

ALGORITHMS; LEARNING; MATHEMATICS; PRINCIPAL COMPONENT ANALYSIS; STOCHASTIC PROCESSES;

EID: 77955914272     PISSN: 01290657     EISSN: None     Source Type: Journal    
DOI: 10.1142/S0129065710002437     Document Type: Article

References (38)

1. E. Oja, A simplified neuron model as principal component analyzer, J. Math. Biol. 15 (1982) 267-273.
2. T. D. Sanger, Optimal unsupervised learning in a single-layer linear feedforward neural network, Neural Networks 2(6) 459-473 (1989).
3. K. I. Diamantaras and S. Y. Kung, Principal Component Neural Networks. Theory and Applications (John Wiley & Sons, 1996).
4. K. I. Diamantaras and S.-Y. Kung, Cross-correlation neural-network models, IEEE Trans. on Signal Processing 42(11) (1994) 3218-3223.
5. S. Fiori, Singular value decomposition learning on double Stiefel manifolds, Int. J. Neural Syst. 13(2) (2003) 1-16.
6. R. Möller and H. Hoffmann, An extension of neural gas to local PCA, Neurocomputing 62 (2004) 305-326.
7. P. Comon and G. H. Golub, Tracking a few extreme singular values and vectors in signal processing, Proc. of the IEEE 78(8) (1990) 1327-1343.
8. G. H. Golub and C. F. van Loan, Matrix Computations, 3rd edn. (Baltimore and London: Johns Hopkins University Press, 1996).
9. G. H. Golub and C. Reinsch, Singular value decomposition and least squares solutions, Numer. Math. 14(5) (1970) 403-420.
10. R. Möller and A. Könies, Coupled principal component analysis, IEEE Trans. on Neural Networks 15(1) (2004) 214-222.
11. R. Möller, First-order approximation of Gram- Schmidt orthonormalization beats deflation in coupled PCA learning rules, Neurocomputing 69(13-15) (2006) 1582-1590.
12. P. F. Baldi and K. Hornik, Learning in linear neural networks: A survey, IEEE Trans. on Neural Networks 6(4) (1995) 837-858.
13. T. D. Sanger, Two iterative algorithms for computing the singular value decomposition from input/output samples, in Adv. Neur., in J. D. Cowan, G. Tesauro and J. Alspector (eds.) 6 (Morgan Kaufmann Publishers, Inc., 1994), pp. 144-151.
14. K. I. Diamantaras and S.-Y. Kung, Multilayer neural networks for reduced-rank approximation, IEEE Trans. on Neural Networks 5(5) (1994) 684-697.
15. D.-Z. Feng, Z. Bao and X.-D. Zhang, A crossassociative neural network for SVD of nonsquared data matrix in signal processing, IEEE Trans. on Neural Networks 12(5) (2001) 1215-1221.
16. A. Weingessel and K. Hornik, SVD algorithms: APEX-like versus subspace methods, Neural Process. Lett. 5 (1997) 177-184.
17. A. Weingessel, An Analysis of Learning Algorithms in PCA and SVD Neural Networks, PhD thesis, Technische Universität Wien (1999).
18. U. Helmke and J. B. Moore, Singular-value decomposition via gradient and self-equivalent flows, Linear Algebra Appl. 169 (1992) 223-248.
19. D.-Z. Feng, X.-D. Zhang and Z. Bao, A neural network learning for adaptively extracting crosscorrelation features between two high-dimensional data streams, IEEE Trans. on Neural Networks 15(6) (2004) 1541-1554.
20. M. Hasan, A logarithmic cost function for principal singular component analysis, in IEEE Int. Conf. on Acoust. Speech Signal Process., 1933-1936 (2008).
21. B. Widrow and M. E. Hoff, Adaptive switching circuits, in IRE WESCON Convention Record, Part 4, 96-104 (1960).
22. S. Haykin, Neural Networks. A Comprehensive Foundation, 2nd edn. (Prentice Hall, 1999).
23. D. E. Rumelhart, G. E. Hinton and R. J. Williams, Learning representations by back-propagating errors, Nature 323(1) (1986) 533-536.
24. P. F. Baldi and K. Hornik, Neural networks and principal component analysis: learning from examples without local minima, Neural Networks 2(1) (1989) 53-58.
25. R. W. Brockett, Dynamical systems that sort lists, diagonalize matrices and solve linear programming problems, in Proc. IEEE Conf. on Decis. Cont. (1988) 799-803.
26. E. Oja, Principal components, minor components, and linear neural networks, Neural Networks 5(6) (1992) 927-935.
27. Y. Miao and Y. Hua, Fast subspace tracking and neural network learning by a novel information criterion, IEEE Trans. on Signal Processing 64(7) (1998) 1967-1979.
28. S. T. Smith, Dynamical systems that perform the singular value decomposition, Syst. Control Lett. 16(5) (1991) 319-327.
29. G. Hori, A general framework for SVD flows and joint SVD flows, in Proc. IEEE Int. Conf. on Acoust. Speech Signal Process. 2 (2003) 693-696.
30. M. A. Hasan, Low-rank approximations with applications to principal singular component learning systems, in Proc. IEEE Conf. on Decis. Contr. (2008) 3293-3298.
31. A. Cichocki, Neural network for singular value decomposition, Electron. Lett. 28(8) (1992) 784-786.
32. F.-L. Luo, R. Unbehauen and Y.-D. Li, Real-time computation of singular vectors, Appl. Math. Comput. 86(2-3) (1997) 197-214.
33. E. K. P. Chong and S. H. Zak, An Introduction to Optimization, 2nd edn. (Wiley, 2001).
34. E. Oja and J. Karhunen, On stochastic approximation of the eigenvectors and eigenvalues of the expectation of a random matrix, J. Math. Anal. Appl. 106(1) (1985) 69-84.
35. H. J. Ritter, T. M. Martinetz and K. J. Schulten, Topology-conserving maps for learning visuo-motorcoordination, Neural Networks 2(3) (1989) 159-168.
36. T. Kohonen, Self-Organizing Maps, Springer Series in Information Sciences (Berlin, Heidelberg: Springer, 1995).
37. T. M. Martinetz, S. G. Berkovich and K. J. Schulten, 'Neural-gas' network for vector quantization and its application to time-series prediction, IEEE Trans. on Neural Networks 4(4) (1993) 558-569.
38. R. A. Horn and C. R. Johnson, Matrix Analysis (Cambridge University Press, 1999).