Convergence analysis of a deterministic discrete time system of Oja's PCA learning algorithm

IEEE Transactions on Neural Networks

Volumn 16, Issue 6, 2005, Pages 1318-1328

  Yi, Zhang ^a   Ye, Mao ^a   Lv, Jian Cheng ^a   Tan, Kok Kiong ^b  

^a UNIVERSITY OF ELECTRONIC SCIENCE AND TECHNOLOGY OF CHINA   (China)

^b NATIONAL UNIVERSITY OF SINGAPORE   (Singapore)

Author keywords

Eigenvalue; Eigenvector; Neural network; Oja's learning algorithm; Principal component analysis (PCA)

Indexed keywords

COMPUTER SIMULATION; CONVERGENCE OF NUMERICAL METHODS; EIGENVALUES AND EIGENFUNCTIONS; LEARNING SYSTEMS; MATRIX ALGEBRA; PRINCIPAL COMPONENT ANALYSIS;

CONVERGENCE ANALYSIS; DETERMINISTIC DISCRETE TIME (DDT) SYSTEMS; OJA LEARNING ALGORITHM;

LEARNING ALGORITHMS;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; ARTIFICIAL NEURAL NETWORK; AUTOMATED PATTERN RECOGNITION; COMPUTER SIMULATION; MATHEMATICAL COMPUTING; METHODOLOGY; NONLINEAR SYSTEM; PRINCIPAL COMPONENT ANALYSIS; SIGNAL PROCESSING; STATISTICAL MODEL;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; COMPUTER SIMULATION; MODELS, STATISTICAL; NEURAL NETWORKS (COMPUTER); NONLINEAR DYNAMICS; NUMERICAL ANALYSIS, COMPUTER-ASSISTED; PATTERN RECOGNITION, AUTOMATED; PRINCIPAL COMPONENT ANALYSIS; SIGNAL PROCESSING, COMPUTER-ASSISTED;

EID: 28244482544     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2005.852236     Document Type: Article

References (25)

1. H. M. Abbas and M. M. Fahmy, "Neural model for Karhunen-Loeve transform with application to adaptive image compression," Inst. Elect. Eng. Proc. I, vol. 140, no. 2, pp. 135-143, Apr. 1993.
2. S. Bannour and R. M. Azimi-Sadjadi, "Principal component extraction using recursive least squares learning," IEEE Trans. Neural Netw., vol. 6, no. 2, pp. 457-469, Mar. 1995.
3. C. Chatterjee, Z. Kang, and V. P. Roychowdhury, "Algorithms for accelerated convergence of adaptive PCA," IEEE Trans. Nerual Netw., vol. 11, no. 2, pp. 338-355, Mar. 2000.
4. T. Chen, Y. Hua, and W. Yan, "Global convergence of Oja's subspace algorithm for principal component extraction," IEEE Trans. Nerual Netw., vol. 9, no. 1, pp. 58-67, Jan. 1998.
5. A. Cichocki, W. Kasprezak, and W. Skarbek, "Adaptive learning algorithm for principal component analysis with partial data," Proc. Cybernetics Syst., vol. 2, pp. 1014-1019, 1996.
6. K. Hornik and C. M. Kuan, "Convergence analysis of local feature extraction algorithm," Neural Netw., vol. 5, pp. 229-240, 1992.
7. S. Y. Kung, K. I. Diamantaras, and J. S. Taur, "Adaptive principal component extraction (APEX) and applications," IEEE Trans. Signal Process., vol. 42, no. 5, pp. 1202-1217, May 1994.
8. L. Ljung, "Analysis of recursive stochastic algorithms," IEEE Trans. Autom. Control, vol. AC-22, no. 4, pp. 551-575, Aug. 1977.
9. E. Oja, "A simplified neuron mode as a principal component analyzer," J. Math. Biol., vol. 15, pp. 167-273, 1982.
10. E. Oja and J. Karhunen, "On stochastic approximation of the eigenvector and eigenvalues of the expectation of a random matrix," J. Math. Anal. App., vol. 106, no. 1, pp. 69-84, Feb. 1985.
11. E. Oja, "Principal components, minor components, and linear neural networks," Neural Netw., vol. 5, pp. 927-935, 1992.
12. S. Ouyang, Z. Bao, and G. S. Liao, "Robust recursive least squares learning algorithm for principal component analysis," IEEE Trans. Nerual Netw., vol. 11, no. 1, pp. 215-221, Jan. 2000.
13. T. D. Sanger, "Optimal unsupervised learning in a single-layer linear feedforward neural network," Neural Netw., vol. 2, pp. 459-473, 1989.
14. A. Tale and G. Cirrincione, "Against the convergence of the minor component analysis neurons," IEEE Trans. Nerual Netw., vol. 10, no. 1, pp. 207-210, Jan. 1999.
15. A. Weingessel and K. Hornik, "Local PCA algorithms," IEEE Trans. Neural Netw., vol. 11, no. 6, pp. 1242-1250, Nov. 2000.
16. W. Y. Yan, U. Helmke, and J. B. Moore, "Global analysis of Oja's flow for neural networks," IEEE Trans. Neural Netw., vol 5, no. 5, pp. 674-683, Sept. 1994.
17. Q. Zhang, "On the discrete-time dynamics of a PCA learning algorithm," Neurocomput., vol. 55, pp. 761-769, Oct. 2003.
18. Q. Zhang and Z. Bao, "Dynamical systems for computing the eigenvectors associated with the largest eigenvalue of a positive definite matrix," IEEE Trans. Neural Netw., vol. 6, no. 3, pp. 790-791, May 1995.
19. Q. Zhang and Y. W. Leung, "Energy function for one-unit Oja algorithm," IEEE Trans. Nerual Netw., vol. 6, no. 5, pp. 1291-1293, Sep. 1995.
20. Q. Zhang and Y. W. Leung, "A class of learning algorithms for principal component analysis and minor component analysis," IEEE Trans. Neural Netw., vol. 11, no. 2, pp. 529-533, Mar. 2000.
21. P. J. Zufiria, "On the discrete-time dynamics of the basic Hebbian neural-network nods," IEEE Trans. Neural Netw., vol. 13, no. 6, pp. 1342-1352, Nov. 2002.
22. Z. Yi, K. K. Tan, and T. H. Lee, "Multistability analysis for recurrent neural networks with unsaturating piecewise linear transfer functions," Neural Computat., vol. 15, pp. 639-662, Mar. 2003.
23. Z. Yi and K. K. Tan, "Dynamical stability conditions for Lotka-Volterra recurrent neural networks with delays," Phys. Rev. E, vol. 66, Jul. 2002.
24. Z. Yi and K. K. Tan, Convergence Analysis of Recurrent Neural Networks. Norwell, MA: Kluwer, 2004.
25. Z. Yi and K. K. Tan, "Multistability analysis of discrete recurrent neural networks with unsaturating piecewise linear transfer functions," IEEE Trans. Neural Netw., vol. 15, no. 2, pp. 329-336, Mar. 2004.