Leap-frog-type learning algorithms over the Lie group of unitary matrices

Neurocomputing

Volumn 71, Issue 10-12, 2008, Pages 2224-2244

  Fiori, Simone ^a  

^a UNIVERSITÀ POLITECNICA DELLE MARCHE   (Italy)

Author keywords

Leap frog type learning steps; Lie group of unitary matrices; Manifold projection; Matrix decompositions (QR, SVD, EVD)

Indexed keywords

EIGENVALUES AND EIGENFUNCTIONS; INTERPOLATION; NEURAL NETWORKS; SINGULAR VALUE DECOMPOSITION;

LEAP-FROG-TYPE NEURAL LEARNING ALGORITHMS; LIE GROUP; PROJECTION METHODS; UNITARY MATRICES;

LEARNING ALGORITHMS;

ACCURACY; ANALYTICAL ERROR; ARTICLE; ARTIFICIAL NEURAL NETWORK; AUTOMATED PATTERN RECOGNITION; BLIND SOURCE SEPARATION; CONCEPTUAL FRAMEWORK; ELECTROENCEPHALOGRAPHY; FUNCTIONAL MAGNETIC RESONANCE IMAGING; HYPOTHESIS; LEARNING ALGORITHM; PRIORITY JOURNAL; PROCESS OPTIMIZATION; SIGNAL PROCESSING; TELECOMMUNICATION;

EID: 44649088387     PISSN: 09252312     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.neucom.2007.09.018     Document Type: Article

References (41)

1. Akuzawa T., and Murata N. Multiplicative nonholonomic/Newton-like algorithm. Chaos Solitons Fractals 12 4 (2001) 785-793
2. Amari S.-i. Natural gradient works efficiently in learning. Neural Comput. 10 (1998) 251-276
3. Anemuller J., Sejnowski T.J., and Makeig S. Complex independent component analysis of frequency-domain EEG data. Neural Networks 16 (2003) 1311-1323
4. Bingham E., and Hyvärinen A. A fast fixed-point algorithm for independent component analysis of complex valued signals. Int. J. Neural Syst. 10 1 (2000) 1-8
5. Calhoun V.D., Adali T., Pearlson G.D., van Zijl P.C.M., and Pekar J.J. Independent component analysis of fMRI data in the complex domain. Magn. Reson. Med. 48 (2002) 180-192
6. Cardoso J.F., and Laheld B. Equivariant adaptive source separation. IEEE Trans. Signal Process. 44 12 (1996) 3017-3030
7. Celledoni E., and Fiori S. Neural learning by geometric integration of reduced 'rigid-body' equations. J. Comput. Appl. Math. 172 2 (2004) 247-269
8. Chefd'hotel C., Tschumperlé D., Deriche R., and Faugeras O.D. Regularizing flows for constrained matrix-valued images. J. Math. Imaging Vision 20 1-2 (2004) 147-162
9. Cichocki A., Umbehauen R., and Rummert E. Robust learning for blind separation of signals. Electron. Lett. 30 17 (1994) 1386-1387
10. Comon P. Independent component analysis, a new concept?. Signal Process. 36 (1994) 287-314
11. Eriksson J., and Koivunen V. Identifiability and separability of linear ICA models revisited. Proceedings of International Conference on Independent Component Analysis and Blind Signal Separation (ICA 2003). Nara, Japan (2003) 23-27
12. Fiori S. A theory for learning by weight flow on Stiefel-Grassman manifold. Neural Comput. 13 7 (2001) 1625-1647
13. Fiori S. A theory for learning based on rigid bodies dynamics. IEEE Trans. Neural Networks 13 3 (2002) 521-531
14. Fiori S. Extended Hebbian learning for blind separation of complex-valued sources. IEEE Trans. Circuits Syst.-Part II 50 4 (2003) 195-202
15. Fiori S. Neural independent component analysis by 'maximum-mismatch' learning principle. Neural Networks 16 8 (2003) 1201-1221
16. Fiori S. Fully-multiplicative orthogonal-group ICA neural algorithm. Electron. Lett. 39 24 (2003) 1737-1738
17. Fiori S. A fast fixed-point neural blind deconvolution algorithm. IEEE Trans. Neural Networks 15 2 (2004) 455-459
18. Fiori S. Non-linear complex-valued extensions of Hebbian learning: an essay. Neural Comput. 17 4 (2005) 779-838
19. Fiori S. Quasi-geodesic neural learning algorithms over the orthogonal group: a tutorial. J. Mach. Learn. Res. 6 (2005) 743-781
20. Fiori S., Albini L., Faba A., Cardelli E., and Burrascano P. Numerical modeling for the localization and the assessment of electromagnetic field sources. IEEE Trans. Magn. 39 3 (2003) 1638-1641
21. Fiori S., and Amari S.-i. Editorial: special issue on "Geometrical methods in neural networks and learning". Neurocomputing 67C (2005) 1-7
22. S. Fiori, T. Tanaka, Committee of Lie-group machines, Technical Report of the Department of Electrical and Electronic Engineering, Tokyo University of Agriculture and Technology, September 2007.
23. Golub G.H., and van Loan C.F. Matrix Computations. third ed. (1996), The John Hopkins University Press, Baltimore, MD
24. Helmke U., and Moore J.B. Optimization and Dynamical Systems (1993), Springer, Berlin
25. Higham N.J. Computing the polar decomposition-with applications. SIAM J. Sci. Stat. Comput. 7 4 (1986) 1160-1174
26. Holm J.E.W., and Botha E.C. Leap-frog is a robust algorithm for training neural networks. Network: Comput. Neural Syst. 10 1 (1999) 1-13
27. Hyvärinen A., Karhunen J., and Oja E. Independent Component Analysis (2001), Wiley, New York
28. Jutten C., and Hérault J. Blind separation of sources, part I: an adaptive algorithm based on neuromimetic architecture. Signal Process. 24 (1991) 1-10
29. Mackey N., Higham N.J., Mackey D.S., and Tisseur F. Functions preserving matrix groups and iterations for the matrix square root. SIAM J. Matrix Anal. Appl. 26 3 (2005) 849-877
30. McLachlan R., and Quispel R. Six lectures on geometric integration. In: DeVore R., Iserles A., and Süli E. (Eds). Foundations of Computational Mathematics (2001), Cambridge University Press, Cambridge 155-210
31. Pajunen P., and Karhunen J. Least-squares methods for blind source separation based on nonlinear PCA. Int. J. Neural Syst. 8 5-6 (1997) 601-612
32. Park H., Amari S.-i., and Lee Y. An information geometrical approach on plateau problems in multilayer perceptron learning. J. KISS (B): Software Appl. 26 4 (1999) 546-556
33. Park H., Murata N., and Amari S.-i. Improving generalization performance of natural gradient learning using optimized regularization by NIC. Neural Comput. 16 2 (2004) 355-382
34. Rattray M., and Saad D. Incorporating curvature information into on-line learning. In: Saad D. (Ed). On-line Learning in Neural Networks (1998), Cambridge University Press, Cambridge
35. Rattray M., and Saad D. Analysis of natural gradient descent for multilayer neural networks. Phys. Rev. E 59 4 (1999) 4523-4532
36. Rattray M., Saad D., and Amari S.-i. Natural gradient descent for on-line learning. Phys. Rev. Lett. 81 (1998) 5461-5464
37. Shalvi O., and Weinstein E. Super-exponential methods for blind deconvolution. IEEE Trans. Inf. Theory 39 2 (1993) 504-519
38. Shen H., and Hüper K. Geometric optimisation and FastICA algorithms. Proceedings of the Mathematical Theory of Networks & Systems Symposium (MTNS 2006). Kyoto, Japan (July 24-28, 2006)
39. Sudjianto A., and Hassoun M.H. Statistical basis of non-linear Hebbian learning and application to clustering. Neural Networks 8 5 (1995) 707-715
40. Theis F. Uniqueness of complex and multidimensional independent component analysis. Signal Process. 84 5 (2004) 951-956
41. Yamada I., and Ogura N. Adaptive projected subgradient method for asymptotic minimization of sequence of nonnegative convex functions. Numer. Funct. Anal. Optim. 25 7-8 (2004) 593-617