Equations of states in singular statistical estimation

Neural Networks

Volumn 23, Issue 1, 2010, Pages 20-34

  Watanabe, Sumio ^a  

^a TOKYO INSTITUTE OF TECHNOLOGY   (Japan)

Author keywords

[No Author keywords available]

Indexed keywords

APRIORI; ASYMPTOTIC NORMALITY; FISHER INFORMATION MATRICES; GENERALIZATION ERROR; GENERALIZATION PERFORMANCE; HIDDEN VARIABLE; HIERARCHICAL STRUCTURES; INFORMATION CRITERION; LEARNING MACHINES; MATHEMATICAL RELATION; MAXIMUM LIKELIHOOD ESTIMATOR; PARAMETRIC MODELS; POSTERIORI DISTRIBUTION; STATISTICAL ESTIMATION; STATISTICAL MODELS; TRAINING ERRORS;

ERRORS; ESTIMATION; FISHER INFORMATION MATRIX; MAXIMUM LIKELIHOOD ESTIMATION; NORMAL DISTRIBUTION;

EQUATIONS OF STATE;

ANALYTICAL ERROR; ARTICLE; BAYES THEOREM; MACHINE LEARNING; MATHEMATICAL ANALYSIS; MAXIMUM LIKELIHOOD METHOD; PARAMETRIC TEST; PREDICTION; PRIORITY JOURNAL; PROBABILITY; STATISTICAL ANALYSIS; STATISTICAL MODEL;

ARTIFICIAL INTELLIGENCE; BAYES THEOREM; COMPUTER SIMULATION; GENERALIZATION (PSYCHOLOGY); HUMANS; MODELS, STATISTICAL; NONLINEAR DYNAMICS; PREDICTIVE VALUE OF TESTS; REFERENCE VALUES; REGRESSION ANALYSIS;

EID: 70649087949     PISSN: 08936080     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.neunet.2009.08.002     Document Type: Article

References (26)

1. Amari S., Park H., and Ozeki T. Singularities affect dynamics of learning in neuromanifolds. Neural Computation 18 5 (2006) 1007-1065
2. Aoyagi M., and Watanabe S. Stochastic complexities of reduced rank regression in Bayesian estimation. Neural Networks 18 7 (2005) 924-933
3. Aoyagi M., and Watanabe S. Resolution of singularities and generalization error with Bayesian estimation for layered neural network. IEICE Transactions J88-D-II 10 (2005) 2112-2124
4. Atiyah M.F. Resolution of singularities and division of distributions. Communications in Pure and Applied Mathematics 13 (1970) 145-150
5. Hagiwara K. On the problem in model selection of neural network regression in overrealizable scenario. Neural Computation 14 8 (2002) 1979-2002
6. Hartigan J.A. A failure of likelihood asymptotics for normal mixture. Proc. of Berkeley Conf. in honor of Jerzy Neyman and Jack Keifer 2 (1985) 807-810
7. Hayasaka T., Kitahara M., and Usui S. On the asymptotic distribution of the least-squares estimators in unidentifiable models. Neural Computation 16 1 (2004) 99-114
8. Hironaka H. Resolution of singularities of an algebraic variety over a field of characteristic zero. Annals of Mathematics 79 (1964) 109-326
9. Kashiwara M. B-functions and holonomic systems. Inventions Mathematicae 38 (1976) 33-53
10. Kollár J. Lectures on resolution of singularities (2007), Princeton University Press, Princeton
11. Nagata K., and Watanabe S. Exchange Monte Carlo sampling from Bayesian Posterior for singular learning machines. IEEE Transactions on Neural Networks 19 7 (2008) 1253-1266
12. Van der Vaart A.W., and Wellner J.A. Weak convergence and empirical processes (1996), Springer
13. Watanabe, S. (1995). Generalized Bayesian framework for neural networks with singular Fisher information matrices. In Proc. of international symposium on nonlinear theory and its applications (pp. 207-210)
14. Watanabe S. Algebraic analysis for singular statistical estimation. Proc. of international journal of algorithmic learning theory. Lecture Notes on Computer Sciences 1720 (1999) 39-50
15. Watanabe S. Algebraic information geometry for learning machines with singularities. Advances in Neural Information Processing Systems (2000) 329-336
16. Watanabe S. Algebraic analysis for nonidentifiable learning machines. Neural Computation 13 4 (2001) 899-933
17. Watanabe S. Algebraic geometrical methods for hierarchical learning machines. Neural Networks 14 8 (2001) 1049-1060
18. Watanabe S. Learning efficiency of redundant neural networks in Bayesian estimation. IEEE Transactions on Neural Networks 12 6 (2001) 1475-1486
19. Watanabe S., and Amari S. Learning coefficients of layered models when the true distribution mismatches the singularities. Neural Computation 15 5 (2003) 1013-1033
20. Watanabe S. Algebraic geometry of singular learning machines and symmetry of generalization and training errors. Neurocomputing 67 (2005) 198-213
21. Watanabe S. Algebraic geometry and learning theory (2006), Morikita publishing in Japanese
22. Watanabe, S. (2007). Generalization and training errors in Bayes and Gibbs estimations in singular learning machines. IEICE Technical Report, NC2007-75 (pp. 25-30)
23. Watanabe, S. (2009). On a relation between a limit theorem in learning theory and singular fluctuation, IEICE Technical Report, NC2008-111 (pp. 45-50)
24. Yamazaki K., and Watanabe S. Singularities in mixture models and upper bounds of stochastic complexity. Neural Networks 16 7 (2003) 1029-1038
25. Yamazaki K., and Watanabe S. Singularities in complete bipartite graph-type Boltzmann machines and upper bounds of stochastic complexities. IEEE Transactions on Neural Networks 16 2 (2005) 312-324
26. Yamazaki K., and Watanabe S. Algebraic geometry and stochastic complexity of hidden Markov models. Neurocomputing 69 (2005) 62-84