Equations of states in statistical learning for an unrealizable and regular case

IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences

Volumn E93-A, Issue 3, 2010, Pages 617-626

  Watanabe, Sumio ^a  

^a TOKYO INSTITUTE OF TECHNOLOGY   (Japan)

Author keywords

Bayes learning; Generalization; Information criterion; Regular; Singular

Indexed keywords

LEARNING SYSTEMS;

BAYES LEARNING; GENERALIZATION; INFORMATION CRITERION; REGULAR; SINGULAR;

EQUATIONS OF STATE;

EID: 77950809473     PISSN: 09168508     EISSN: 17451337     Source Type: Journal    
DOI: 10.1587/transfun.E93.A.617     Document Type: Article

References (26)

1. H. Akaike, "A new look at the statistical model identification, " IEEE Trans. Autom. Control, vol. 19, pp. 716-723, 1974.
2. S.-i. Amari, H. Park, and T. Ozeki, "Singularities affect dynamics of learning in neuromanifolds, " Neural Comput., vol. 18, no. 5, pp. 1007-1065, 2006.
3. M. Aoyagi and S. Watanabe, "Stochastic complexities of reduced rank regression in Bayesian estimation, " Neural Netw., vol. 18, no. 7, pp. 924-933, 2005.
4. K. Hagiwara, "On the problem in model selection of neural network regression in overrealizable scenario, " Neural Comput., vol. 14, no. 8, pp. 1979-2002, 2002.
5. J. A. Hartigan, "A failure of likelihood asymptotics for normal mixture, " Proc. Barkeley Conf. in Honor of Jerzy Neyman and Jack Keifer, vol. 2, pp. 807-810, 1985.
6. T. Hayasaka, M. Kitahara, and S. Usui, "On the asymptotic distribution of the least-squares estimators in unidentifiable models, " Neural Comput., vol. 16, no. 1, pp. 99-114, 2004.
7. N. Murata, S. Yoshizawa, and S. Amari, "Network information criterion - Determining the number of hidden units for an artificial neural network model, " IEEE Trans. Neural Netw., vol. 5, no. 6, pp. 865-872, Nov. 1994.
8. K. Nagata and S. Watanabe, "Asymptotic behavior of exchange ratio in exchange monte carlo method, " International Journal of Neural Neworks, vol. 21, no. 7, pp. 980-988, 2008.
9. K. Nagata and S. Watanabe, "Exchange monte carlo sampling from bayesian posterior for singular learning machines, " IEEE Trans. Neural Netw., vol. 19, no. 7, pp. 1253-1266, 2008.
10. K. Takeuchi, "Distribution of statistic and criterion for optimal model selection, " Surikagaku, vol. 153, pp. 12-18, 1976.
11. S. Watanabe, "Generalized Bayesian framework for neural networks with singular Fisher information matrices, " Proc. International Symposium on Nonlinear Theory and Its applications, pp. 207-210, Las Vegas, 1995.
12. S. Watanabe, "Algebraic analysis for singular statistical estimation, " Proc. International Journal of Algorithmic Learning Theory, Lecture Notes on Computer Sciences, 1720, pp. 39-50, 1999.
13. S. Watanabe, "Algebraic analysis for nonidentifiable learning machines, " Neural Computation, vol. 13, no. 4, pp. 899-933, 2001. (Pubitemid 33594310)
14. S. Watanabe, "Algebraic geometrical methods for hierarchical learning machines, " Neural Netw., vol. 14, no. 8, pp. 1049-1060, 2001. (Pubitemid 32846604)
15. S. Watanabe, "Learning efficiency of redundant neural networks in Bayesian estimation, " IEEE Trans. Neural Netw., vol. 12, no. 6, pp. 1475-1486, 2001.
16. S. Watanabe and S.-I. Amari, "Learning coefficients of layered models when the true distribution mismatches the singularities, " Neural Comput., vol. 15, no. 5, pp. 1013-1033, 2003.
17. S. Watanabe, "Algebraic geometry of singular learning machines and symmetry of generalization and training errors, " Neurocomputing, vol. 67, pp. 198-213, 2005.
18. S. Watanabe, "Equations of states in statistical estimation, " arXiv:0712.0653, 2007.
19. S. Watanabe, "Generalization and training errors of Bayes and Gibbs estimation in singular learning machines, " IEICE Technical Report, NC2007-75, 2007.
20. S. Watanabe, "A formula of equations of states in singular learning machines, " Proc. WCCI, pp. 2098-2105, 2008.
21. S. Watanabe, "A limit theorem in singular regression problem, " arXiv:0901.2376v1, 2009.
22. S. Watanabe, Algebraic geometry and statistical learning theory, Cambridge University Press, 2009.
23. S. Watanabe, "On a relation between a limit theorem in learning theory and singular fluctuation, " IEICE Technical Report, NC2008-111, 2009.
24. K. Yamazaki and S. Watanabe, "Singularities in mixture models and upper bounds of stochastic complexity, " International Journal of Neural Networks, vol. 16, no. 7, pp. 1029-1038, 2003.
25. K. Yamazaki and S. Watanabe, "Singularities in complete bipartite graph-type Boltzmann machines and upper bounds of stochastic complexities, " IEEE Trans. Neural Netw., vol. 16, no. 2, pp. 312-324, 2005.
26. K. Yamazaki and S. Watanabe, "Algebraic geometry and stochastic complexity of hidden Markov models, " Neurocomputing, vol. 69, pp. 62-84, 2005. (Pubitemid 41644424)