Quasi-geodesic neural learning algorithms over the orthogonal group: A tutorial

Journal of Machine Learning Research

Volumn 6, Issue , 2005, Pages

  Fiori, Simone ^a  

^a UNIVERSITY OF PERUGIA   (Italy)

Author keywords

Differential geometry; Diffusion type gradient; Lie groups; Non negative independent component analysis; Riemannian gradient

Indexed keywords

APPROXIMATION THEORY; DIFFERENTIAL EQUATIONS; GEODESY; INDEPENDENT COMPONENT ANALYSIS; INTEGRAL EQUATIONS; NEURAL NETWORKS; NUMERICAL METHODS; OPTIMIZATION;

DIFFERENTIAL GEOMETRY; DIFFUSION-TYPE GRADIENT; LIE GROUPS; NON-NEGATIVE INDEPENDENT COMPONENT ANALYSIS; RIEMANNIAN GRADIENT;

LEARNING ALGORITHMS;

EID: 21844443579     PISSN: 15337928     EISSN: None     Source Type: Journal    
DOI: None     Document Type: Article

References (29)

1. S.-i. Amari. Differential-Geometrical Methods in Statistics. Lecture Notes in Statistics 28, Springer-Verlag, 1989.
2. S.-i. Amari. Natural gradient works efficiently in learning. Neural Computation, 10:251-276, 1998.
3. rd International Conference on Independent Component Analysis and Blind Signal Separation pages 114-119, San Diego, California, USA, 2001
4. E. Celledoni and S. Fiori. Neural learning by geometric integration of reduced 'rigid-body' equations. Journal of Computational and Applied Mathematics, 172(2):247-269, 2004.
5. A. Cichocki and S.-i. Amari. Adaptive Blind Signal and Image Processing, J. Wiley & Sons, 2002
6. S. Fiori. A theory for learning by weight flow on Stiefel-Grassman manifold. Neural Computation, 13(7):1625-1647, 2001.
7. S. Fiori. A theory for learning based on rigid bodies dynamics. IEEE Trans. on Neural Networks, 13(3):521-531, 2002.
8. S. Fiori. A fast fixed-point neural blind deconvolution algorithm. IEEE Trans. on Neural Networks, 15(2):455-459, 2004.
9. S. Fiori. Non-linear complex-valued extensions of Hebbian learning: An essay. Neural Computation, 17(4):779-838, 2005.
10. S. Fiori. Formulation and integration of learning differential equations on the Stiefel manifold. IEEE Trans. on Neural Networks, forthcoming.
11. S. Fiori and S.-i. Amari. Editorial: Special issue on "Geometrical Methods in Neural Networks and Learning", Neurocomputing, forthcoming.
12. W. K. Hastings. Monte Carlo sampling methods using Markov chains and their applications. Biometrika, 57:97-109, 1970.
13. A. Hyvärinen, J. Karhunen and E. Oja. Independent Component Analysis, John Wiley & Sons, 2001.
14. D. J. Higham. An algorithmic introduction to numerical simulation of stochastic differential equations. SIAM Review, 43(3):525-546, 2001.
15. R. E. Kass, B. P. Carlin, A. Gelman and R. M. Neal. Markov Chain Monte Carlo in practice: A roundtable discussion. The American Statistician, 52(2):93-100, 1998.
16. N. Keshavaand J. F. Mustard. Spectral unmixing. IEEE Signal Processing Magazine, 19(1):44-57, 2002.
17. X. Liu, A. Srivastava and K. Gallivan. Optimal linear representation of images for object recognition. IEEE Trans. on Pattern Analysis and Machine Intelligence, 26(5):662-666, 2004.
18. N. Metropolis, A. W. Rosenbluth, M. N. Rosenbluth, A. H. Teller and E. Teller. Equations of state calculations by fast computing machines. Journal of Chemical Physics, 21:1087-1091, 1953.
19. Y. Nishimori. Learning algorithm for ICA by geodesic flows on orthogonal group. In Proc. of the International Joint Conference on Neural Networks pages 1625-1647, 1999.
20. P. J. Olver. Applications of Lie groups to differential equations. Graduate Texts in Mathematics 107, Second Edition, Springer, 2003.
21. H. Park, S.-i. Amari and K. Fukumizu. Adaptive Natural Gradient Learning Algorithms for Various Stochastic Models. Neural Networks, 13:755-764, 2000.
22. M. D. Plumbley. Conditions for non-negative independent component analysis. IEEE Signal processing Letters, 9(6):177-180, 2002.
23. M. D. Plumbley. Algorithms for nonnegative independent component analysis. IEEE Trans. on Neural Networks, 14(3):534-543, 2003.
24. M. D. Plumbley. Lie group methods for optimization with orthogonality constraints. In Proceedings of the International Conference on Independent Component Analysis and Blind Signal Separation, pages 1245-1252, Granada, Spain, 2004.
25. A. Srivastava, U. Grenander, G. R. Jensen and M. I. Miller. Jump-diffusion Markov processes on orthogonal groups for object recognition. Journal of Statistical Planning and Inference, 103(1/2):15-37, 2002.
26. H. H. Yang and S.-i. Amari. Adaptive online learning algorithms for blind separation: Maximum entropy and minimum mutual information. Neural Computation, 9:1457-1482, 1997.
27. G. R. Warnes. The normal kernel coupler: An adaptive MCMC method for efficiently sampling from multi-modal distributions. Technical Report 39, Dept. of Statistics, University of Washington, 2001.
28. D. R. Wilson and T. R. Martinez. The general inefficiency of batch training for gradient descent learning. Neural Networks, 16(10):1429-1451, 2003.
29. L.-Q. Zhang, A. Cichocki and S.-i. Amari. Geometrical structures of FIR manifold and multichannel blind deconvolution. Journal of VLSI for Signal Processing Systems, 31:31-44, 2002.