Modulated Hebb-Oja Learning rule - A method for principal Subspace analysis

IEEE Transactions on Neural Networks

Volumn 17, Issue 2, 2006, Pages 345-356

  Jankovic, Marko V ^a   Ogawa, Hidemitsu ^b  

^a UNIVERSITY OF BELGRADE   (Serbia)

^b TOKYO INSTITUTE OF TECHNOLOGY   (Japan)

Author keywords

Learning algorithm; Neural networks; Principal component analysis (PCA); Principal subspace analysis (PSA)

Indexed keywords

COMPUTATIONAL METHODS; LEARNING ALGORITHMS; LEARNING SYSTEMS; PRINCIPAL COMPONENT ANALYSIS;

LEARNING ALGORITHM; LINEAR MAPPING; PRINCIPAL SUBSPACE ANALYSIS (PSA);

NEURAL NETWORKS;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; ARTIFICIAL NEURAL NETWORK; AUTOMATED PATTERN RECOGNITION; COMPUTER SIMULATION; DECISION SUPPORT SYSTEM; EVALUATION; MATHEMATICAL COMPUTING; METHODOLOGY; THEORETICAL MODEL;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; COMPUTER SIMULATION; DECISION SUPPORT TECHNIQUES; MODELS, THEORETICAL; NEURAL NETWORKS (COMPUTER); NUMERICAL ANALYSIS, COMPUTER-ASSISTED; PATTERN RECOGNITION, AUTOMATED;

EID: 33644909037     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2005.863455     Document Type: Review

References (38)

1. K. Abed-Meraim, S. Attallah, A. Ckheif, and Y. Hua, "Orthogonal Oja algorithm," IEEE Signal Process. Lett., vol. 7, no. 5, pp. 116-119, May 2000.
2. S.-I. Amari, "Neural theory of assosiation and concept formation," Biolo. Cybern., vol. 26, pp. 175-185, 1977.
3. P. Baldi and K. Hornik, "Learning in linear neural networks: A survey," IEEE Trans. Neural Netw., vol. 6, no. 4, pp. 837-858, Jul. 1995.
4. T.-P. Chen and S. Amari, "Unified stabilization approach to principal and minor components," Neural Networks, vol. 14, pp. 1377-1387, 2001.
5. T. Chen, Y. Hua, and W. Yan, "Global convergence of Oja's subspace algorithm for principal component extraction," IEEE Trans. Neural Netw., vol. 9, no. 1, pp. 58-67, Jan. 1998.
6. A. Chichocki and S.-I. Amari, Adaptive Blind Signal and Image Processing - Learning Algorithms and Applications. New York: Wiley, 2003.
7. I. D. Coope and P. F. Renaud, "Trace inequalities with applications to orthogonal regression and matrix nearness problems," Canterbury Univ., Canterbury, U.K., Res. Rep. UCDMS2000/17, 2000.
8. G. Deco and D. Obradovic, An Information-Theoretic Approach to Neural Computing. New York: Springer-Verlag, 1996.
9. K. I. Diamantaras, "Robust principal component extracting neural networks," in ICNN'96, Washington, DC, 1996, pp. 74-77.
10. S. C. Douglas, S. Y. Kung, and S. Amari, "A self-stabilized minor subspace rule," IEEE Signal Process. Lett., vol. 5, no. 12, pp. 328-330, Dec. 1998.
11. S. Fiori, "A theory for learning by weight flow on Stiefel-Grassman manifold," Neural Comput., vol. 13, no. 7, pp. 1625-1647, Jul. 2001.
12. P. Földiák, "Adaptive network for optimal linear feature extraction," in IJCNN'89, Washington, DC, 1989, pp. 401-405.
13. C. Fyfe and R. J. Baddeley, "Finding compact and sparse distributed representations of visual images," Network: Comput. Neural Syst., vol. 6, no. 3, pp. 334-344, 1995.
14. C. Fyfe and R. J. Baddeley, "Nonlinear data structure extraction using simple Hebbain networks," Biol. Cybern., vol. 72, no. 6, pp. 533-541, 1995.
15. M. C. Goodall, "Performance of stochastic net," Nature, vol. 185, pp. 557-558, 1960.
16. G. F. Harpur and R. W. Prager, "Experiments with simple Hebbian-based learning rules in pattern classification tasks," Eng. Dept., Cambridge Univ., Cambridge, U.K., Tech. Rep. CUED/F-INFENG/TR 168, 1994.
17. G. F. Harpur, "Low entropy coding with unsupervised neural networks," Ph.D. dissertation, Eng. Dept., Cambridge Univ., Cambridge, UK, 1997.
18. M. Jankovic, "Anew simple ∞ OH neuron model as a biologically plausible principal component analyzer," IEEE Trans. Neural Netw., vol. 14, no. 4, pp. 853-859, Jul. 2003.
19. M. Jankovic and H. Ogawa, "A new modulated hebb learning rule - Biologically plausible method for local computation of principal subspace," Int. J. Neural Syst., vol. 13, no. 4, pp. 215-224, 2003.
20. L. Ljung, "Analysis of recursive stochastic algorithms," IEEE Trans. Autom. Control, vol. AC-22, no. 8, pp. 551-575, Aug. 1977.
21. R. Möller and A. Konies, "Coupled principal component analysis," IEEE Trans. Neural Netw., vol. 15, no. 1, pp. 214-222, Jan. 2004.
22. H. Ogawa, "Karhunen-Loéve subspace," in Proc. Int. Conf. Pattern Recognition, Hague, The Netherlands, 1992, pp. 75-78.
23. E. Oja, "A simplified neuron model as a principal component analyzer," J. Math. Biol., vol. 15, pp. 267-273, 1982.
24. E. Oja, Subspace Method of Pattern Recognition. New York: Wiley, 1983.
25. E. Oja, "Neural networks, principal components, and subspaces," Int. J. Neural Syst., vol. 1, pp. 61-68, 1989.
26. E. Oja and J. Karhunen, "On stochastic approximation of the eigenvectors and eigenvalues of the expectation of a random matrix," J. Math. Anal., Appl., vol. 106, pp. 69-84, 1985.
27. E. Oja, H. Ogawa, and J. Wangviwattana, "Principal component analysis by homogeneous neural networks," IEICE Trans. Inf. Syst., pt. Part I: The weighted subspace criterion, vol. E75-D, no. 3, pp. 366-375, 1992.
28. E. Oja, H. Ogawa, and J. Wangviwattana, "Principal component analysis by homogeneous neural networks, Part II: Analysis and extensions of the learning algorithm," IEICE Trans. Inf. Syst., vol. E75-D, no. 3, pp. 376-382, 1992.
29. S. Ouyang, Z. Bao, and G. Liao, "Robust recursive least squares learning algorithm for principal component analysis," IEEE Trans. Neural Netw., vol. 11, no. 1, pp. 215-221, Jan. 2000.
30. J. Peltonen and S. Kaski, "Discriminative components of data," IEEE Trans. Neural Netw., vol. 16, no. 1, pp. 68-83, Jan. 2005.
31. M. D. Plumbley, "On information theory and unsupervised neural networks," Eng. Dept., Cambridge Univ., Cambridge, UK, Tech. Rep. CUED/F-INFENG/TR. 78, 1991.
32. M. D. Plumbley and E. Oja, "A "nonnegative PCA" algorithm for independent component analysis," IEEE Trans. Neural Netw., vol. 15, no. 1, pp. 66-76, Jan. 2004.
33. K. Waheed and F. M. Salem, "Blind information-theoretic multiuser detection algorithms for DS-CMDA and WCDMA downlink systems," IEEE Trans. Neural Netw., vol. 16, no. 4, pp. 937-948, Jul. 2005.
34. A. Weingessel and K. Hornik, "Local PCA algorithms," IEEE Trans. Neural Netw., vol. 11, no. 6, pp. 1242-1250, Nov. 2000.
35. L. Xu, "Least mean square error reconstruction principle for self-organizing neural nets," Neural Networks, vol. 6, pp. 627-648, 1993.
36. W. Yan, U. Helmke, and J. B. Moore, "Global analysis of Oja's flow for neural networks," IEEE Trans. Neural Netw., vol. 5, no. 5, pp. 674-683, Sep. 1994.
37. Q. Zhang and Y. Leung, "A class of learning algorithms for principal component analysis and minor component analysis," IEEE Trans. Neural Netw., vol. 11, no. 2, pp. 529-533, Mar. 2005.
38. Y. Zhang, J. Weng, and W.-S. Hwang, "Auditory learning: A developmental method," IEEE Trans. Neural Netw., vol. 16, no. 3, pp. 601-616, Mar. 2000.