Asymptotic distributions associated to Oja's learning equation for neural networks

IEEE Transactions on Neural Networks

Volumn 9, Issue 6, 1998, Pages 1246-1257

  Delmas, Jean Pierre ^b   Cardoso, Jean François ^a,c  

^a IEEE

^b INSTITUT NATIONAL DES TÉLÉCOMMUNICATIONS   (France)

^c TELECOM PARISTECH   (France)

Author keywords

Adaptive estimation; Eigenvectors; Oja's learning equation; Principal component analysis; Subspace estimation

Indexed keywords

COMPUTER SIMULATION; EIGENVALUES AND EIGENFUNCTIONS; ESTIMATION; MATRIX ALGEBRA; NEURAL NETWORKS; ORDINARY DIFFERENTIAL EQUATIONS;

ADAPTIVE ESTIMATION; ASYMPTOTIC DISTRIBUTIONS; LEARNING EQUATION; PRINCIPAL COMPONENT ANALYSIS; SUBSPACE ESTIMATION;

LEARNING ALGORITHMS;

EID: 0032208194     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/72.728373     Document Type: Article

References (31)

1. H. Krim and M. Viberg, "Two decades of array signal processing research," IEEE Signal Processing Mag., July 1996, pp. 67-94.
2. E. Moulines, P. Duhamel, J. F. Cardoso, and S. Mayrargue, "Subspace methods for blind identification of multichannel FIR filters," IEEE Trans. Signal Processing, vol. 43, pp. 516-525, Feb. 1995.
3. E. Oja, "Principal components, minor components and linear neural networks," Neural Networks, vol. 5, pp. 927-935, 1992.
4. C. M. Kuan and K. Hornik, "Convergence of learning algorithms with constant learning rates," IEEE Trans. Neural Networks, vol. 2, pp. 484-489, Sept. 1991.
5. K. Hornik and C. M. Kuan, "Convergence analysis of local feature extraction algorithms," Neural Networks, vol. 5, pp. 229-240, 1992.
6. L. Ljung, "Analysis of recursive stochastic algorithms," IEEE Trans. Automat. Contr., vol. AC-22, pp. 551-575, 1977.
7. H. J. Kushner and D. S. Clark, Stochastic Approximation Methods for Constrainted and Unconstrainted Systems. New York: Springer-Verlag, 1978.
8. H. J. Kushner, Approximation and Weak Convergence Methods for Random Processes. Cambridge, MA: MIT Press, 1984.
9. A. Benveniste, M. Métivier, and P. Priouret, Adaptive Algorithms and Stochastic Approximations. Berlin, Germany: Springer-Verlag, 1990.
10. J. P. Delmas, "Performance analysis of parameterized adaptive eigensubpace algorithms," in Proc. ICASSP, Detroit, MI, May 1995, pp. 2056-2059.
11. B. Yang and F. Gersemsky, "Asymptotic distribution of recursive subspace estimators," in Proc. ICASSP, Atlanta, GA, May 1996, pp. 1764-1767.
12. J. P. Delmas and F. Alberge, "Asymptotic performance analysis of subspace adaptive algorithms introduced in the neural-network literature," IEEE Trans. Signal Processing, vol. 46, pp. 170-182, Jan. 1998.
13. J. P. Delmas and J. F. Cardoso "Performance analysis of an adaptive algorithm for tracking dominant subspaces," to appear IEEE Trans. Signal Processing, Nov. 1998.
14. R. Williams, "Feature discovery through error-correcting learning," Univ. California, San Diego, Inst. Cognitive Sci., Tech. Rep. 8501, 1985.
15. P. Baldi, "Linear learning: Landscapes and algorithms," in Proc. NIPS, Denver, CO, 1988.
16. E. Oja, "Neural networks, principal components and subspaces," Int. J. Neural Syst., vol. 1, no. 1, pp. 61-68, 1989.
17. E. Oja and J. Karhunen, "On stochastic approximation of the eigenvectors and eigenvalues of the expectation of a random matrix," J. Math. Anal. Applicat., vol. 106, pp. 69-84, 1985.
18. H. Rutishauser, "Computational aspects of F. L. Bauer's simultaneous iteration method," Numer. Math., vol. 13, pp. 4-13, 1969.
19. E. Oja, "A simplified neuron model as a principal components analyzer," J. Math. Biol., vol. 15, pp. 267-273, 1982.
20. P. Baldi and K. Hornik, "Backpropagation and unsupervised learning in linear networks," in Y. Chauvin and D. E. Rummelhart, Eds., Backpropagation: Theory, Architectures, and Applications. Hillside, NJ: Lawrence Earlbaum, 1991.
21. A. Krogh and J. A. Hertz, "Hebbian learning of principal components," in R. Eckmiller, G. Hartmann, and G. Hauske, Eds., Parallel Processing in Neural Systems and Computers. Amsterdam, The Netherlands: Elsevier, 1990.
22. W. Y. Yan, U. Helmke, and J. B. Moore, "Global analysis of Oja's flow for neural networks," IEEE Trans. Neural Networks, vol. 5, pp. 674-683, Sept. 1994.
23. T. Chen, Y. Hua, and W-Y. Yan, "Global convergence of Oja's subspace algorithm for principal component extraction," IEEE Trans. Neural Networks, vol. 9, pp. 58-67, Jan. 1998.
24. L. Russo, "An outer product neural network for extracting principal components from a time series," in B. H. Juang et al., Eds., Neural Networks for Signal Processing. New York: IEEE Press, 1991, pp. 161-170.
25. B. Yang, "Projection approximation subspace tracking," IEEE, Trans. Signal Processing, vol. 43, pp. 95-107, Jan. 1995.
26. _, "Convergence analysis of the subspace tracking algorithms PAST and PASTd," in Proc. ICASSP, Atlanta, GA, May 1996, pp. 1760-1763.
27. E. Oja, H. Ogawa, and J. Wangviwattana, "Principal component analysis by homogeneous neural networks, Part I: The weighted subspace criterion," IEICE Trans. Inform. Syst., vol. E75-D,3, pp. 366-375, 1992.
28. _, "Principal component analysis by homogeneous neural networks, Part II: Analysis and extensions of the learning algorithms," IEICE Trans. Inform. Syst., vol. E75-D,3, pp. 376-382, 1992.
29. N. L. Owsley, "Adaptive data orthogononalization," Proc. IEEE Int. Conf. ASSP, Apr. 1978, pp. 109-112.
30. T. W. Anderson, An Introduction to Multivariate Statistical Analysis, 2nd ed. New York: Wiley, 1984.
31. T. Kato, Perturbation Theory for Linear Operators. Berlin, Germany: Springer-Verlag, 1995.