Global convergence of Oja's PCA learning algorithm with a non-zero-approaching adaptive learning rate

Theoretical Computer Science

Volumn 367, Issue 3, 2006, Pages 286-307

  Cheng Lv, Jian ^a   Yi, Zhang ^a   Tan, K K ^b  

^a UNIVERSITY OF ELECTRONIC SCIENCE AND TECHNOLOGY OF CHINA   (China)

^b NATIONAL UNIVERSITY OF SINGAPORE   (Singapore)

Author keywords

Deterministic discrete time system; Global convergence; Oja's PCA learning algorithm; Principal component analysis (PCA)

Indexed keywords

ADAPTIVE SYSTEMS; CONVERGENCE OF NUMERICAL METHODS; DISCRETE TIME CONTROL SYSTEMS; ONLINE SYSTEMS; PRINCIPAL COMPONENT ANALYSIS;

ADAPTIVE LEARNING; DETERMINISTIC DISCRETE TIME (DDT) SYSTEMS; GLOBAL CONVERGENCE; OJA'S PCA LEARNING ALGORITHMS;

LEARNING ALGORITHMS;

EID: 33750743749     PISSN: 03043975     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.tcs.2006.07.012     Document Type: Article

References (31)

1. Abbas H.M., and Fahmy M.M. Neural model for Karhunen-Loeve transform with application to adaptive image compression. IEE Proc.-I 140 2 (1993) 135-143
2. Anisse T., and Gianalvo C. Against the convergence of the minor component analysis neurons. IEEE Trans. Neural Network 10 1 (1999) 207-210
3. Bannour S., and Azimi-Sadjadi R. Principal component extraction using recursive least squares learning. IEEE Trans. Neural Networks 6 2 (1995) 457-469
4. Chatterjee C. Adaptive algorithms for first principal eigenvector computation. Neural Networks 18 2 (2005) 145-159
5. Chatterjee C., Kang Z., and Roychowdhury V.P. Algorithms for accelerated convergence of adaptive PCA. IEEE Trans. Neural Networks 11 2 (2000) 338-355
6. Chatterjee C., Roychowdhury V.P., and Chong E.P. On relative convergence properties of principal component analysis algorithm. IEEE Trans. Neural Networks 9 2 (1998) 319-329
7. Chen T., Hua Y., and Yan W. Global convergence of Oja's subspace algorithm for principal component extraction. IEEE Trans. Neural Networks 9 1 (1998) 58-67
8. Chen H., and Liu R.W. An on-line unsupervised learning machine for adaptive feature extraction. IEEE Trans. Circuits Systems-II: Analog Digital Signal Process. 41 2 (1994) 87-98
9. Cichocki A., Kasprzak W., and Skarbek W. Adaptive learning algorithm for principal component analysis with partial data. Cybernetics Systems 2 (1996) 1014-1019
10. Cirrincione G., Cirrincione M., Hérault J., and Huffel S.V. The MCA EXIN neuron for the minor component analysis. IEEE Trans. Neural Networks 13 1 (2002) 160-186
11. Haykin S. Neural Network, A Comprehensive Foundation. second ed. (2002), Prentice-Hall Inc., Englewood Cliffs, NJ
12. Hornik K., and Kuan C.M. Convergence analysis of local feature extraction algorithm. Neural Networks 5 (1992) 229-240
13. Jankovic M.V. A new simple ∞OH neuron model as a biologically plausible principal component analyzer. IEEE Trans. Neural Networks 14 4 (2003) 853-859
14. Kung S., Diamantaras K., and Taur J. Adaptive principal component extraction (APEX) and applications. IEEE Trans. Signal Process. 42 5 (1994) 1202-1217
15. Möller R., and Könies A. Coupled principal component analysis. IEEE Trans. Neural Networks 15 1 (2004) 214-222
16. Oja E. A simplified neuron mode as a principal component analyzer. J. Math. Biol. 15 (1982) 167-273
17. Oja E. Principal components, minor components, and linear neural networks. Neural Networks 5 (1992) 927-935
18. Oja E., and Karhunen J. On stochastic approximation of the eigenvector and eigenvalues of the expectation of a random matrix. J. Math. Anal. Appl. 106 1 (1985) 69-84
19. Ouyang S., Bao Z., and Liao G.S. Robust recursive least squares learning algorithm for principal component analysis. IEEE Trans. Neural Networks 11 1 (2000) 215-221
20. Sanger T.D. Optimal unsupervised learning in a single-layer linear feedforward neural network. Neural Networks 2 (1989) 459-473
21. Vegas J.M., and Zufiria P.J. Generalized neural networks for spectral analysis: dynamics and Liapunov functions. Neural Networks 17 2 (2004) 233-245
22. Weingessel A., and Hornik K. Local PCA algorithms. IEEE Trans. Neural Networks 11 5 (2000) 1242-1250
23. Yan W.Y., Helmke U., and Moore J.B. Global analysis of Oja's flow for neural networks. IEEE Trans. Neural Networks 5 4 (1994) 674-683
24. Yi Z., and Tan K.K. Convergence Analysis of Recurrent Neural Networks (2004), Kluwer Academic Publishers, Dordrecht
25. Yi Z., Ye M., Lv J.C., and Tan K.K. Convergence analysis of a deterministic discrete time system of Oja's PCA learning algorithm. IEEE Trans. Neural Networks 16 6 (2005) 1318-1328
26. Yu W., Poznyak A.S., and Li X. Multiplier dynamic neural networks for non-linear system on-line identification. Internat. J. Control 74 18 (2001) 1859-1864
27. Zhang Q. On the discrete-time dynamics of a PCA learning algorithm. Neurocomputing 55 (2003) 761-769
28. Zhang Q., and Bao Z. Dynamical system for computing the eigenvectors associated with the largest eigenvalue of a positive definite matrix. IEEE Trans. Neural Networks 6 (1995) 790-791
29. Zhang Q., and Leund Y.W. Energy function for one-unit Oja algorithm. IEEE Trans. Neural Networks 6 (1995) 1291-1293
30. Zhang Q., and Leund Y.W. A class of learning algorithm for principal component analysis and minor component analysis. IEEE Trans. Neural Networks 11 2 (2000) 529-533
31. Zufiria P.J. On the discrete-time dynamic of the basic Hebbian neural-network nods. IEEE Trans. Neural Networks 13 6 (2002) 1342-1352