A formula of equations of states in singular learning machines

Proceedings of the International Joint Conference on Neural Networks

Volumn , Issue , 2008, Pages 2098-2105

  Watanabe, Sumio ^a  

^a TOKYO INSTITUTE OF TECHNOLOGY   (Japan)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; EDUCATION; EQUATIONS OF STATE; ERRORS; FISHER INFORMATION MATRIX; LEARNING SYSTEMS; NEURAL NETWORKS;

A POSTERIORI; ASYMPTOTIC NORMALITIES; COMPUTATIONAL INTELLIGENCES; FISHER INFORMATIONS; GENERALIZATION ERRORS; GENERALIZATION PERFORMANCES; LEARNING MACHINES; MAXIMUM LIKELIHOOD ESTIMATORS; PRIORI DISTRIBUTIONS; STATISTICAL MODELS; TRAINING ERRORS;

NORMAL DISTRIBUTION;

EID: 56349163383     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/IJCNN.2008.4634086     Document Type: Conference Paper

References (32)

1. S.-i. Amari, H. Park, and T. Ozeki, "Singularities Affect Dynamics of Learning in Neuromanirblds," Neural Comput., 18(5),pp.1007 -1065, 2006.
2. M.Aoyagi, S.Watanabe,"Stochastic complexities of reduced rank regression in Bayesian estimation," Neural. Networks, Vol. 18.No.7, pp.924-933, 2005.
3. M.Aoyagi,S.Watanabe,"Resolution of singularities and generalization error with Bayesian estimation for layered neural network," Vol.J88-D-II, No.10, pp.2112-2124, 2005.
4. M.F.Atiyah,"Resolution of singularities and division of distributions," Comm. Pure Appl. Math., Vol.13,pp.145-150, 1970.
5. K. Hagiwara, "On the Problem in Model Selection of Neural Network Regression in Overrealizable Scenario, " Neural Comput., Vol.14,Vol.8, pp.1979 - 2002, 2002.
6. J.A.Hartigan,"A failure of likelihood asymptotics for normal mixture," Proc. of Barkeley Conf. in honor of Jerzy Neyman and Jack Keifer, Vol.2, pp.807-810, 1985.
7. T. Hayasaka, M. Kitahara, and S. Usui, "On the Asymptotic Distribution of the Least-Squares Estimators in. Unidentifiable Models," Neural Comput., Vol.16, No.1, pp.99 - 114, 2004.
8. H. Hironaka, "Resolution of singularities of an algebraic variety over a field of characteristic zero," Ann. of Math., Vol.79, 109-326,1964.
9. M. Kashiwara, "B-functions and holonomic systems," Inventions Math., 38, 33-53.1976.
10. J. Kollár,"Lectures on Resolution of Singularities," Princeton University Press, (Princeton), 2007.
11. Kenji Nagata and Sumio Watanabe,"Exchange Monte Carlo Sampling from Bayesian Posterior for Singular Learning Machines", to appear in IEEE Transactions on Neural Networks.
12. Kenji Nagata, Sumio Watanabe, "Asymptotic Behavior of Exchange Ratio in Exchange Monte Carlo Method", International Journal of Neural. Networks, to appear.
13. S. Nakajima, S. Watanabe, "Variational Bayes Solution of Linear Neural Networks and its Generalization Performance," Neural Computation, to appear.
14. Y. Nishiyama, S.Watanabe, "Asymptotic Behavior of Stochastic Complexity of Complete Bipartite Graph-type Boltzmann Machines," Proc. of ICONIP2006, (Chaina,HongKong) to appear, 2006.
15. K.Watanabe, S.Watanabe, "Stochastic Complexities of Gaussian Mixtures in Variational Bayesian Approximation," Journal of Machine Learning Research, Vol.7, (Apr), pp. 625-644, 2006.
16. K.Watanabe, S. Watanabe, "Stochastic complexities of general, mixture models in variational. Bayesian leaning," Neural Networks, to appear.
17. S.Watanabe, "Generalized Bayesian framework for neural networks with singular Fisher information matrices," Proc. of International Symposium on Nonlinear Theory and Its applications, (Las Vegas), pp.207-210, 1995.
18. S.Watanabe, "Algebraic analysis for singular statistical estimation," Proc. of International Journal of Algorithmic Learning Theory, Lecture Notes on Computer Sciences, 1720, pp.39-50, 1999.
19. S. Watanabe, "Algebraic information geometry for learning machines with singularities, " Advances in Neural Information Processing Systems, (Denver, USA), pp.329-336. 2001.
20. S.Watanabe, "Algebraic Analysis for Nonidentifiable Learning Machines," Neural Computation, Vol.13, No.4, pp.899-933, 2001.
21. S. Watanabe, "Algebraic geometrical methods for hierarchical learning machines," Neural Networks, Vol.14, No.8,pp. 1049-1060, 2001.
22. S. Watanabe, "Learning efficiency of redundant neural networks in Bayesian estimation," IEEE Transactions on Neural Networks, Vol. 12, No.6, 1475-1486, 2001.
23. S.Watanabe, S.-I.A:mari,"Learning coefficients of layered models when the true distribution mismatches the singularities", Neural Computation, Vol.15,No.5,1013-1033, 2003.
24. S.Watanabe,"Algebraic geometry of singular learning machines and symmetry of generalization and training errors," Neurocomputing, Vol.67,pp.198-213,2005.
25. S. Watanabe,"Algebraic geometry and learning theory, " Morikita publishing, 2006.
26. S. Watanabe,"Almost all learning machines are singular," Proc. of international Symposimum on IEEE FOCI 2007.
27. S.Watanabe,"Generalizarion and training errors in Bayes and Gibbs estimations in singular learning machines," IEICE technical report (NC2007-75), Vol.2007-12, pp.25-30, December, 2007.
28. S.Watanabe,"Equations of States in Singular Statistical Estimation," arXiv:0712.653, 5th December. 2007.
29. A. W. van der Vaart, Jon A. Wellner, "Weak. Convergence and Empirical. Processes," Springer,1996.
30. K.Yamazaki, S.Watanabe,"Singularities in mixture models and upper bounds of stochastic complexity." International Journal of Neural Networks, Vol.16, No.7, pp. 1029-1038, 2003.
31. K.Yamazaki, S.Watanabe," Singularities in Complete bipartite graphtype Boltzmann machines and upper bounds of stochastic complexities", IEEE Trans. on Neural Networks, Vol. 16 (2), pp.312-324, 2005.
32. K. Yamazaki and S. Watanabe, "Algebraic geometry and stochastic complexity of hidden Markov models", Neurocomputing, Vol.69, pp.62-84, 2005.