Differential and algebraic geometry of multilayer perceptrons

IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences

Volumn E84-A, Issue 1, 2001, Pages 31-38

  Amarj, Shun Ichi ^a   Ozeki, Tomoko ^a  

^a RIKEN BRAIN SCIENCE INSTITUTE   (Japan)

Author keywords

[No Author keywords available]

Indexed keywords

ALGEBRA; HIERARCHICAL SYSTEMS; LEARNING SYSTEMS; MATHEMATICAL MODELS; NONLINEAR SYSTEMS; PROBABILITY DISTRIBUTIONS;

ALGEBRAIC GEOMETRY; INFORMATION GEOMETRY;

MULTILAYER NEURAL NETWORKS;

EID: 0035125794     PISSN: 09168508     EISSN: None     Source Type: Journal    
DOI: None     Document Type: Article

References (26)

1. S. Aman, "Theory of adaptive pattern classifiers," IEEE Trans., vol.EC-16, no.3, pp.299-307, 1967.
2. S. Amari, Differential-geometrical method in statistics, Springer, Berlin, 1985.
3. S. Amari, "Dualistic geometry of the manifold of higherorder neurons," Neural Networks, vol.4, pp.443-451, 1991.
4. S. Amari, "A universal theorem on learning curves," Neural Networks, vol.6, no.2, pp.161-166, 1993.
5. S. Amari, "Information geometry of the EM and em algorithms for neural networks," Neural Networks, vol.8, no.9, pp.1379-1408, 1995.
6. S. Amari, "Natural gradient works efficiently in learning," Neural Computation, vol.10, pp.251-276, 1998.
7. S. Amari, A. Cichocki, and H.H. Yang, "A new learning algorithm for blind signal separation," Advances in Neural Information Processing Systems 8, Proc. 1995 Conference NIPS, eds. D.S. Touretzky, M.C. Mozer, and M.E. Hasselmo, pp.757-763.
8. S. Amari, K. Kurata, and H. Nagaoka, "Information geometry of Boltzmann machines," IEEE Trans. Neural Networks, vol.3, no.2, pp.260-271, 1992.
9. S. Amari and N. Murata, "Statistical theory of learning curves under entropie loss criterion," Neural Computation, vol.5, pp.140-153, 1992.
10. S. Amari and H. Nagaoka, Methods of Information Geometry, AMS and Oxford University Press, New York, 2000.
11. S. Amari, H. Park, and K. Fukumizu, "Adaptive method of realizing natural gradient learning for multilayer perceptions," Neural Computation, vol.12, pp.1399-1409, 2000.
12. A.M. Chen, H. Lu, and R. Hecht-Nielsen, "On the geometry of feedforward neural network error surfaces," Neural Computations, vol.5, pp.910-927, 1993.
13. K. Fukumizu, "Generalization error of linear neural networks in unidentifiable cases," Lecture Notes in Computer Sciences, 1720, eds. O. Watanabe and T. Yokomori, pp.SI62, Springer, 1999.
14. K. Fukumizu and S. Amari, "Local minima and plateaus in hierarchical structures of multilayer perceptrons," Neural Networks, vol.13, pp.317-327, 2000.
15. K. Hagiwara, N. Toda, and S. Usui, "Nonuniqueness of connecting weights and AIC in multi-layered neural networks," IEICE Trans., vol J76-D-II, PP.2058-2065, 1993.
16. T. Hayasaka, K. Hagiwara, N. Toda, and S. Usui, "On the estimation and prediction errors of function representation with orthonormal discrete variable basis in regression model," The Brain and Neural Networks, vol.4, pp.18-26, 1997.
17. V. Kurkova and P.C. Kainen., "Functionally equivalent feedforward neural networks," Neural Computation, vol.6, pp.543-558, 1994.
18. H. Park, S. Amari, and K. Fukumizu, "Adaptive natural gradient learning algorithms for various stochastic models," Neural Networks, accepted 2000.
19. G. Pistone and C. Sempi, "An infinite-dimensional geometric structure on the space of all the probability measures equivalent to a given one," Annals of Statistics, vol.23, pp.1543-1561, 1995.
20. G. Pistone and M.P. Rogantin, "The exponential statistical manifold: Mean parameters, orthogonality, and space transformation," Bernoulli, vol.5, pp.721-760, 1999.
21. M. Rattray, D. Saad, and S. Amari, "Natural gradient descent for on-line learning," Physical Review Letters, vol.81, no.24, pp.5461-5464, 1998.
22. S.M. Rüger and A. Ossen, "The metric of weight space," Neural Processing Letters, vol.5, pp.63-72, 1997.
23. D. Saad and S.A. Solla, "On-line learning in soft committee machines," Physical Review E, vol.52, pp.4225-4243, 1995.
24. H.J. Sussmann, "Uniqueness of the weights for minimal feedforward nets with a given input-output map," Neural Networks, vol.5, pp.589-593, 1992.
25. H.H Yang and S. Amari, "Complexity issues in natural gradient descent method for training multilayer perceptrons," Neural Computation, vol.10, pp.2137-2157, 1998.
26. S. Watanabe, "Algebraic analysis for non-identifiable learning machines," Neural Computation, accepted.